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CHAPTER  I 
INTRCDUCTICN 

The  Department  of  Defense  (DCD)  is  faced  with  the 
task  of  acquiring  new  weapon  systems  to  maintain  the  United 
State's  position  as  the  leader  of  the  free  world.  The  acqui- 
sition of  these  new  weapon  systems  has  been  characterized 
by  a history  of  substantial  grov/zh  in  actual  cosus  relative 
to  earlier  estimated  program  costs  ( 1 ; 11  ; 18 ; 25 ) . In  the 
last  decade  this  cost  growth  in  the  acquisition  of  Air  Force 
weapon  systems  by  the  DOD  has  become  a major  problem  for  the 
DOD,  Congress  and  the  American  people  (18).  iharoin  defines 
cost  growth  as  " . . . when  actual  cost  is  greater  than  the 
initial  estimate  for  a program  [l3:8l|-]." 

Another  area  that  causes  chagrin  is  the  disparity 
among  the  cost  estimates  that  are  made  by  many  different 
Air  Force  and  DCD  organizations.  During  his  tenure  as 
Secretary  of  Defense,  Mr.  Laird  emphasized  the  critical 
importance  of  valid  cost  estimates  by  establishing  the 
Office,  Secretary  of  Defense  Cost  Analysis  Improvement  Group 
(CSD/CAIG).  He  stated,  "Service  groups  responsible  for 
independent  estimates  and  the  OSD  Cost  Analysis  Improvement 
Group  should  work  closely  in  developing  uniform  criteria 
for  cost  estimates  Presently  the  Controller  of  the 

Air  Force  (AF/AC)  and  ohe  Director  of  Procurement  Policy 
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(AF/LGP),  Headquarters  USAF  are  making  studies  of  the  dif- 
ferent estimating  groups  so  they  can  understand  and  reduce 
disparities  in  cost  estimates  made  by  the  variety  of  dif- 
ferent organizations  involved  in  estimating  the  costs  of  new 
weapon  systems  (15:  59). 

The  Air  Force  Business  Research  I'ianagement  Center 
has  been  established  to  coordinate  research  particularly 
relevant  to  managing  the  Air  Force  procurement  function. 

It  monitors  the  different  research  efforts  being  accomplished 
to  provide  a focal  point  for  information  and  limit  unnec- 
cessary  duplication.  It  is  the  Center's  firm  belief  that 
very  few,  if  any,  individuals  have  an  overall  perception  of 
how  the  variety  of  major  weapon  system  cost  estimates 
interrelate  and  what  causes  the  wide  variances  that  seem  to 
exist  among  the  different  estimates  made  by  the  different 
organizations  (26)  . 

Statement  of  the  Problem 

There  is  a need  to  better  understand  as  an  overall 
process  how  cost  estimates  for  new  weapon  systems  are 
developed  in  the  Air  Force,  who  develops  them,  and  the 
contribution  each  estimate  is  designed  to  make  in  the 
overall  acquisition  process.  A thorough  analysis  should  help 
explain  why  Air  Force  cost  estimates  for  the  same  weapon 
system  vary  so  much  from  each  other  and  from  the  actual, 
final  cost  of  the  weapon. 
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Background 


The  weapon  sysrem  acquisition  process  begins  when  the 
need  for  a new  weapon  system  is  first  identified,  and  it 
continues  through  the  conceptual,  validation,  development, 
and  production  phases  (I3:l6).  At  many  points  during  the 
process,  different  agencies  are  required  to  make  estim*ates 
of  what  the  new  weapon  system  will  cost  (53).  Most  of  the 
major  cost  estimates  are  used  principally  to  justify  and  to 
support  proposals  to  the  Defense  System  Acquisition  Review 
Council  (DASRC),  which  in  turn  determines  whether  to  proceed 
with  or  terminate  the  weapon  system  development  (11:5). 

The  cost  estimates  are  also  used  to  justify  and  to  support 
proposals  to  Congress  in  efforts  to  obtain  and  retain 
congressional  approval  of  funding.  When  there  is  an 
increase  in  the  cost  of  the  program,  as  measured  by  the 
differences  between  the  development  cost  estimates  and  the 
actual  cost  of  the  program,  the  DCD  must  explain  why  and  is 
frequently  severly  criticized  for  its  inability  to  control 
acquisition  programs  (11:9;  16:5). 

Multiple  estimates  are  made  by  different  agencies 
using  different  techniques  (26;  53).  The  cost  estimates 
provided  by  the  DCD  in  support  of  new  weapon  system  proposals 
frequently  turn  out  to  be  inaccurate  (DTsll-O).  The  subsequent 
cost  overruns  result  in  considerable  Congressional  and 
public  concern  reflected  in  a loss  of  confidence  in  the  DCD 
estimating  ability  and  in  future  program  justification 
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efforts  (2?! ^0-44). 


Justification 

The  DCD,  if  it  is  to  regain  Congressional  confidence 
and  support  for  existing  and  future  weapon  system  programs, 
must  improve  the  cost  estimates  used  to  support  proposals 
brought  before  Congress.  It  is  essential  not  only  to  acquire 
new  weapon  systems  at  minimum  cost,  but  to  accurately  predict 
the  cost  of  these  weapons  when  advocating  them.  Cnly  with 
accurate  estimates  can  the  Air  Force,  DOD,  and  Congress  make 
appropriate  and  logical  cost  tradeoffs  among  the  weapon 
system  programs  competing  for  defense  dollars,  and  thus 
insure  the  most  effective  use  of  the  scarce  U.  S.  tax  dollar 
(22:3!  27:20). 

Mr.  Laird,  while  serving  as  Secretary  of  Defense, 
stated  that  valid  cost  estimates  are  critical  to  a 
successful  defense  posture  (3l)*  Professor  Bruce  Baker 
carried  this  a step  further  in  the  concluding  remarks  of  his 
1972  doctoral  dissertation,  stating,  "It  can  be  surmised 
. . . that  as  original  estimates  become  more  realistic, 
greater  pressure  will  exist  to  stay  within  those  original 
estimates  [ 2:124]."  The  pressure  to  stay  within  the 
original  estimates  and  to  reduce  cost  overruns  is  present 
now,  even  though  these  estimates  may  not  have  become  more 
realistic  or  accurate.  Congress  has  reduced  the  percentage 
of  the  federal  budget  allocated  to  the  DCD  from  60  per  cent 
in  195^  "to  approximately  30  per  cent  in  1974  ( 5:10).  This 
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is  partially  due  to  the  fact  thax  the  American  public  has 
been  demanding  and  receiving  a larger  share  of  the  federal 
budget  for  social  programs  such  as  aid  to  education,  welfare 
and  health  care  ( 8:10),  but  it  may  also  be  due  partially 
to  a perceived  inability  of  the  DCD  to  control  its  weapon 
system  acquisition  process  (21:^9). 

In  197^1  Secretary  of  Defense  James  Schlesinger 
stated  that  the  defense  policy  of  the  United  States  would 
continue  to  be  one  of  maintaining  "...  a reasonable  stable 
level  of  defense  effort  . . . [24:4]."  Thus  it  would  seem 
that  the  DCD  must  continue  to  arm  itself  with  new  v/eapcn 
systems  to  meet  future  threats , but  with  an  apparently 
reduced  buying  power.  To  do  this  it  must  regain  Congres- 
sional confidence  in  its  ability  zo  make  accurate  cost 
estimates . 

After  interviewing  several  personnel  interested  in 
the  estimating  process  (l4j  15  i 26  ),  it  is  evident  thac  a 
variety  of  opinions  exist  as  to  how  the  DCD  estimating 
process  works.  An  analysis  of  their  comments  indicate  that 
there  are  conflicts  regarding  who  makes  cost  estimates  on 
new  weapon  systems  and  what  techniques  are  used  to  make 
these  estimates  at  various  points  in  the  weapon  system 
acquisition  process.  Misconceptions  are  also  evident  in 
regards  to  comparisons  between  cost  esti.mates  made  in  the 
conceptual  phases  and  estimates  made  in  the  production 
phase. 


Ob  .1  active 


The  objective  of  this  study  is  to  clarify  and 
document  the  overall  cost  estimating  process  used  by  the 
Air  Force  in  the  acquisition  of  new  major  weapon  systems. 


Research  Questions 

The  following  research  questions  are  posed  to  g’-iide 
the  research  toward  the  stated  objectives: 

1.  What  model  of  the  cost  estimating  process  for 
new  major  Air  Force  weapon  systems  can  be  developed  that 
depicts 

a.  the  organization/agency  providing  each  cost 

estimate , 

b.  the  purpose  for  which  each  cost  estimate  is 

developed , 

0.  the  point  in  the  acquisition  process  where 
these  estimates  are  made,  and 

d.  the  techniques  which  are  appropriate  to  each 
organization/agency  in  producing  their  estimates,  considering 
the  information  available  at  the  point  when  the  estim.ate  is 
needed? 

2.  If  such  a model  can  be  developed,  can  it  be 
validated  to  demonstrate  that  it  realistically  reflects 
the  "real  world"  estimating  environment  of  the  agencies 
concerned  with  the  acquisition  of  new  major  Air  Force 
weapon  systems? 
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Research  Question  number  1 will  be  answered  by  a 


thorough  literaxure  review  in  Chapter  II.  This  literature 
review  will  include  AF  ;jjid  DCD  regulations,  manuals, 
pamphlets,  directives,  and  other  publications  as  applicable. 

The  methodology  for  validating  the  model  will  be 
presented  in  Chapter  III.  The  answer  to  Research  Question 
number  2 will  be  developed  in  Chapter  IV  using  the 
methodology  presented  in  Chapter  III.  Finally,  Chapter  V 
will  contain  the  conclusions  reached  as  a conseo.uence  of  this 
research  effort  and  the  recommendations  the  authors  have 
for  implementing  the  results  and  for  needed  future  research. 
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CHAPTER  II 


LITERATURE  RF/IE.J 


A literature  review  was  conducted  to  answer  Research 
Question  number  1.  A model  of  the  cost  estimating  process 
used  by  the  Air  Force  in  the  acquisition  of  new  major  weapon 
systems  was  developed.  The  literature  review  covered  the 
appropriate  regulations,  manuals,  direcxives,  and  other 
publications  available  to  defines 

1.  the  Air  Force  weapon  system  acquisition  process 
and  phases ; 

2.  the  sources  of  cost  estimates  made  within  this 


process ; 

3.  the  type  of  estimate  made  or  purpose  for  which 
an  estimate  is  developed  by  each  organization/agency  during 
the  phases  of  the  weapon  system  acquisition  process; 

4.  the  amount  of  data  available  during  the  different 
phases  of  the  weapon  system  acquisition  process  to  make 

cost  estimates;  and 

5.  the  techniques  used  by  the  agencies  to  make  their 
cost  estimates. 

These  five  areas  were  considered  the  major  concerns  in 
developing  a model  of  the  cost  estimating  prooess  for 
acquiring  new  major  Air  Force  weapon  systems.  The  model 
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developed  through  this  literature  review  is  presented  in 
Figure  2 at  the  end  of  this  chapter. 


I 

i 


I 

I 


Weapon  System  Acquisition 
Free  ess 

The  process  by  which  weapon  systems  are  acquired  by 

the  Air  Force  is  termed  the  weapon  system  acquisition  | 

I 

process.  This  weapon  system  acquisition  process  is  composed  i 

of  six  (6)  phases:  (l)  the  conceptual  phase;  (2)  the  }j 

validation  phase:  (3)  the  full  scale  development  phase;  l] 

(4)  the  production  phase;  (5)  "the  deployment  phase;  and 
(6)  the  reutilicaticn  and  disposition  phase  ( 60  :2-3). 

Cost  estimates  dealing  with  system  acquisition  are  norm.ally  .j 

made  in  the  first  four  phases.  Cost  estimates  made  for  j 

optimal  system  deployment  and  for  the  reutilization  and  | 

disposition  of  the  system  are  beyond  the  scope  of  this  • | 

research  effort.  Therefore,  only  the  first  four  phases 

will  be  considered  in  this  study.  (See  Fig-are  1 for  a I 

graphic  presentation  of  the  weapon  33/3 tern,  acquisition  : 

J 

process ) . J 

DOD  Directive  50OO.I  defines  a major  -weapon  system  as  | 

one  -whose  program  dollar  value  will  have  an  estimated  I 

Research  Development  Test  and  Evaluation  cost  in  excess  of  | i 

50  million  dollars  or  an  estimated  prod-action  cost  in  ;j 

excess  of  200  million  dollars  (29  :1).  For  this  research 

project,  only  major  weapon  systems  within  the  weapon  system  i 

acq-iisiticn  process  are  considered. 
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Conceptual  phase.  During  Phe  conceptual  phase,  an 
operational  requirement  for  a new  weapon  system  is 
identified  (11:5).  This  need  is  normally  defined  by  an 
operational  command,  but  others  do  identify  need  requirements. 
Contractors,  military  study  groups,  and  organizations  such 
as  RAND,  Analytical  Systems  Incorporated,  and  MITR2  also 
identify  such  needs  ( 60:10-22).  Regardless  of  who 
initiates  the  identification  of  the  need  for  a new  weapon 
system.,  official  recognition  of  the  requirement  begins  with 
the  preparation  and  submission  of  a Required  Operational 
Capability  (R0C)(  60  : 10-21).  A .RCC  is  submitted  to 
Headquarters,  US.l?  under  the  provisions  of  AFR  57-1  ( 31). 

.IFR  57-1  defi.nes  a RCC  as  "a  formal,  numbered  document, 
used  to  identify  an  operational  need  and  to  request  a new 
or  im.proved  capability  for  the  operating  forces.  The 
capability  sought  is  described  in  terms  of  operational 
objective,  operational  environ.ment , support  and  maintenance 
concepts,  and  concept  of  operation  ^31' 2]."  The  ROC 
provides  the  information  and  supporting  rationale  necessary 
to  develop  a thorough  understanding  of  the  deficiency  to  be 
alleviated  and  the  corrective  action  proposed  ( 60  : 10-27). 

The  Conceptual  phase  has  three  objectives  (12  :2l): 

(l)  to  establish  the  military,  technical,  and  economic  basis 
required  to  support  a decision  on  whether  to  acquire  the 
new  weapon  system:  (2)  to  identify  the  alternative  approaches 
available  and  select  the  preferred  one;  and  (3)  to  gather 
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data  and  make  a detailed  analysis  which  will  support  a 
decision  to  proceed  with  the  acquisition  process.  Portions 
of  this  data  are  used  to  make  cost  estimates  (l2:2i^).  This 
analysis  and  a plan  for  the  program  are  documented  in  a 
Development  Concept  Paper  (DC?)  which  is  reviewed  by  the 
Defense  Systems  Acquisition  Review  Council  (DSARC)(6C  :2-^). 

The  DSARC  was  established  to  provide  the  Secretary 
of  Defense  with  information  and  recommendations  on  the 
status  and  readiness  of  each  major  weapon  system  concept 
in  order  to  proceed  into  the  next  phase  in  the  acquisition 
process  (31:'f).  Tn  the  conceptual  phase,  the  DSARC  must 
make  a decision  (DSARC  I-Program  Decision)  whether  to 
recommend  to  the  Secretary  of  Defense  to  proceed  into  the 
validation  phase  or  to  reject  the  system  as  propcsed  (6C:2-u) 

Validation  phase.  In  the  validation  phase  the 
technical,  schedule,  and  cost  req.uireme.nts  of  the  weapon 
system  are  refined  and  validated  (12:25).  The  objective  of 
the  validation  phase  is  :o  establish  firm  and  realistic 
performance  specifications  which  meet  operational 
requirements  (12:25).  This  phase  is  normally  conducted  by 
the  Air  Force  working  with  two  or  more  contractors  (if 
possible)  who  are  interested  in  performing  the  full  scale 
development  and  production  of  the  weapon  system  (ll:5). 

The  contractors,  working  independently  of  each  other,  present 
design  proposals  for  consideration  by  the  Air  Force  (,11  :V). 


Cnee  again,  a DSARC  decision  (L'SARC  II-Ratif ication 
Decision)  must  be  obtained  in  order  to  proceed  to  the 
development  phase  (12:26).  DSARC  II  reviews  the  program 
with  respect  to  the  technical  risk  involved,  the  time 
required  to  obtain  a completed  weapon  system,  and  the 
estimated  cost  of  the  weapon  system  (11:?). 

Full  scale  develorment  phase.  The  objective  of  the 
full  scale  development  phase  is  a completely  defined  weapon 
system  in  terms  of  technical  performance,  schedule,  and  cost 
(11:5).  A limited  number  of  working  models  (prototypes)  are 
fabricated  during  the  full  scale  development  phase.  These 
hardware  models  are  used  to  provide  actual  performance 
demonstrations  and  verify  the  weapon  system  design.  The 
documentation  necessary  to  produce  the  weapon  system  for 
inventory  is  also  identified  and  is  developed  during  this 
phase  (12:2?). 

The  culmination  of  the  full  scale  development  phase 
is  the  DSARC  III  decision  (production  decision).  The  DC? 
is  updated  and  the  DSARC  determines  whether  or  not  to 
recommend  full  scale  production  and  deployment  of  the 
weapon  system.  It  must  also  determine  the  initial  quantity 
to  be  produced  and  approve  plans  for  future  production 
(12;  23).  DSARC  III  is  the  last  chance  to  stop  the  new 
weapon  system  prior  to  full  scale  production  (f?!^-!). 
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Production  phase.  In  the  production  phase,  the 


weapon  system  and  its  support  equipment  are  produced  for 
operational  use  (12:28).  During  this  phase,  responsibility 
for  support  and  use  of  the  weapon  system  is  gradually 
transferred  from  Air  Force  Systems  Command  (AFSC)  to  AFLC 
for  support  and  to  the  using  command  for  operations. 
Operational  testing  takes  place  during  this  phase. 
Production  and  deployment  of  the  weapon  system  occur 
simultaneously  and  may  continue  over  long  periods  of  time 
(11  : 8). 


Cost  estimates  are  made  in  each  of  the  phases  of 
the  weapon  system  acquisition  process  to  support  proposals 
made  to  the  DSARC . The  DSAP.C  considers  the  validity  of  the 
cost  estimates  in  making  their  decision  on  whether  to 
proceed  to  the  next  phase  or  to  stop  the  program  (31=7). 

The  phases  of  the  weapon  system  acquisition  process  can 
thus  provide  the  basis  for  identifying  when  cost  estimates 
are  made  and  the  amount  and  kind  of  data  available  for 
making  cost  estim.ates  cn  new  weapon  system.s. 


Sources  of  Cost  Estimates 

Four  forms  of  cost  estimates  are  developed  during 

the  weapon  system  acquisition  process.  These  estimates  are 

the  Cost  Analysis  Improvement  Group  (CAIG)  estimates! 

Independent  Cost  Estimates  (ICE);  Air  Force  Systems  Command 

System  Program  Office  (SPC)  estimiates;  and  contractor 

estimates  or  proposals  (I9=l). 
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Cost  Analysis  Improvement  Group.  The  CAIG  was 
established  under  the  office  of  the  Secretary  of  Defense 
^ (OSD)  by  DOD  Directive  $000.^.  The  primaby  responsibility 

^ of  the  CAIG  is  to  " . . . act  as  an  advisory  body  to  the 

DSARC  on  matters  relating  to  cost  [31=  2’’."  The  CAIG  provides 
the  DSARC  with: 

1.  a review  and  evaluation  of  independent  cost 
analysis  provided  by  the  MAJCCM  performing  the  Independent 
Cost  Estimate  and  program  cost  estimates  provided  by  AESC , 
to  include  all  elements  of  system  costs; 

2.  criteria,  standards,  and  procedures  concerning 

' preparation  and  presentation  of  cost  estimates;  and 

3.  an  assessment/recommendation  on  cost  objectives 
prior  to  including  them  in  approved  DCFs  (31:2). 

The  CAIG  is  also  responsible  for  developing  methods , 
techniques,  and  policies  to  improve  cost  estimating  by  DOD 
components  and  to  resolve  issues  arising  over  the 
comparability  and  completeness  of  cost  data  (31:2). 

The  OSD  CAIG  reviews  and  evaluates  all  cost 


estimates  prior  zo  their  submission  to  the  DSARC  for  review 
( 60  :10-iJ-6).  This  review  and  evaluation  talces  place  at  the 
completion  of  the  conceptual,  validation  and  full  scale 
development  phases  of  weapon  system  acquisition  (60  :2-4). 
The  CSD  CAIG  resolves  any  differences  between  the  ICE  and 
the  SFO  estimates.  In  this  context,  it  provides  an  estimate 
of  its  own  to  the  DSARC  ( 3I  :2). 
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Air  Force  Systems  Command.  AFSC  I<ianual  173-1  estab- 
lishes the  requirements  for  the  Aeronautical  Systems  Division 
(ASD) , Electronics  Systems  Division  (E3D) , and  the  Space  & 
Missile  Systems  Organisation  (SAIviSO)  to  make  cost  estimates 
in  support  of  planning  and  program/budgeting  activities 
associated  with  the  concept  formulation,  development,  and 
acquisition  of  new  weapon  systems  (33:1.1). 

AjSD  has  primary  responsibility  for  planning, 
developing,  and  acquiring  aeronautical  systems  including 
aircraft  and  air  launched  missile  weapon  systems.  ESD  has 
primary  responsibility  for  planning,  developing,  and 
acquiring  all  major  electronic  systems.  ' SAMSG  has  the 
primary  responsibility  for  planning,  developing,  and 
acquiring  all  ballistic  missile  and  space  systems  ( 45:7-9)* 
These  organizations  establish  a System  Program  Office  (5P0) 
to  monitor  and  manage  each  weapon  system.  One  principal 
responsibility  of  each  SPG  is  to  develop  an  official  program 
cost  estimate  for  AFSC  (42). 

Cost  estimates  by  the  appropriate  system  program 
office  are  prepared  at  any  time  throughout  the  weapon  system 
acquisition  process,  but  are  mandatory  at  three  significant 
points  in  the  process.  These  points  are  just  prior  to  the 
three  DSARC  milestones  and  in  preparation  for  the  DSARC 
review  and  decision  (33:1-4). 

Independent  Cost  Estimate.  Air  Foroe  Regulation 
173-11  establishes  the  procedures  for  the  Independent  Cost 
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Analysis  (ICA)  Program  (35).  An  ICA  is  a cost  estimate  made 
independent  of  the  official  (SPO)  program  cost  estimate. 

The  purpose  of  the  Independent  Coso  Estimate  (ICE)  is  to 
provide  a test  of  the  reasonableness  of  the  SPC  program 
estimate.  To  insure  that  the  ICE  is  not  a repeat  of  the  SPC 
estimate,  the  techniques  used  to  prepare  the  ICE  must  be 
different  than  those  used  to  prepare  the  SPO  estimate  (35*1). 

KQ  USAF,  Directorate  of  Management  Analysis 
designates  which  major  command  will  have  the  responsibi.lity 
for  preparing  the  ICE.  The  designated  major  command  provides 
the  team  members,  as  required.  It  must  also  insure  that  the 
SPC  provides  the  program,  contract,  technical,  and  cost  data 
required  by  the  team  to  perform  the  ICA  and  make  the  ICE 
(35:1).  The  team  preparing  the  ICE  must  be  organizationally 
separated  from  the  SPO  managing  the  system.  No  member 
participating  in  the  preparation  of  the  ICE  may  have 
participated  in  the  preparation  of  the  SPC  estimate  fcr  the 
same  weapon  system  (35*l). 

An  ICE  is  prepared  on  a major  weapon  system  for  each 
DSARC  milestone.  The  major  command  designated  to  perform 
the  ICE  is  notified  approximately  65  workdays  prior  to  the 
DSARC  review.  The  results  of  the  ICE  are  presented  to  the 
OSD  CAIG  ten  workdays  prior  to  the  DSARC  review  (35*3). 

Ma.icr  commands.  APR  57-1  establishes  the 
procedures  and  assigns  the  responsibilities  for  writing 
RCCs  for  new  and  improved  operational  capabilities  (60:10-20). 


It  tasks  AFSC  and/or  AFLC  with  the  responsibility  to  prepare 
budget  cost  inforination.  Budget  cost  information  is  defined 
as  "Information  on  proposed  alternative  solutions,  costs, 
and  schedules  for  satisfying  a RCC  [34:2]." 

AFSC  is  specifically  tasked  to  provide  preliminary 
cost  estimates,  to  include  life  cycle  cost  projections  if 
available,  for  those  RCCs  requiring  a research  or  development 
effort  (34:3).  AFLC  is  specifically  tasked  to  provide  cost 
estimates  for  those  RCCs  that  can  be  satisfied  by  modifi- 
cation to  a configuration  item  (34:3). 

Contractor  proposals/estimates.  The  defense  industry 
is  composed  of  approximately  22, COO  prime  con'ractcrs  and 
100,000  subcontractors  {22-57)‘  These  contractors  are  used 
throughout  the  weapon  system  acquisition  process  ~o  support 
the  Air  Force  efforts  in  designing  and  estimating  the  cost 
of  new  weapon  systems  as  well  as  producing  the  weapon 
system  (13:22-33).  They  provide  technical,  feasibility, 
and  cost  studies  to  the  Air  Force  in  the  conceptual  phase 
and  cost  proposals,  budget  estimates  and  cost  performance 
reports  in  the  conceptual,  validaticn,  and  full  scale 
development  phases.  Cost  estimates  of  the  program's  progress 
and  compliance  with  earlier  estimates,  in  the  form  of  cost 
performance  reports,  are  provided  in  the  production  phase 
( 18: 22-36;  53;  60:10-34).  These  cost  estimates  are  used 
by  the  appropriate  SFC  as  a basis  for  their  cost  estimates 
and  in  tradeoff  studies  to  determine  v/hich  alternative 


system  to  develop  to  satisfy  the  operational  requirement 
and/or  which  design  to  pursue  (53;  60 : 15-33 . 3^) • 


Amount  of  Data  Available 

Professor  Ba2<er,  in  his  doctoral  dissertation, 
"Improving  Cost  Estimating  and  Analysis  in  DCD  and  NASA", 
identified  Historical  Data  Problems  as  a major  problem 
area  for  estimations.  One  question  in  a survey  conducted 
for  his  dissertation  requested  information  about  the 
principle  problems  of  cost  estimating  perceived  by  persons 
in  the  field.  Twenty  three  per  cent  of  the  1353  individuals 
involved  in  making  cost  estimates  indicated  that  problems 
related  to  the  amount  of  historical  data  available  were  the 
most  important  problems  of  cost  estimating  and  analysis. 

A?£C  reponses  to  the  survey  indicated  26;^  (7^  out  of  285) 
felt  that  data  availability  was  the  most  important  problem 

(2: 50-5^) • 


Data  P 


curren 


operat 


The  problems  grouped  into  the  category  of  Historical 
roblems  were: 

1.  data  availability  and  collection  problems! 

2.  lack  of  accurate,  reliable,  credible,  valid,  and 
^ c a t a ! 

3.  lack  of  data  base  and/or  computerised  data  bank; 
insufficient  data  regarding  installing  and 

ing  costs;  and 


5.  insufficient  data  regarding  recurring  vs. 
non-recurring  costs  (2:  51)* 
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Problems  involving  techniques,  tools,  methodology, 
and  procedures  were  also  identified  as  major  problems  of  cost 
estimating  by  of  the  respondents.  The  specific  problems 
identified  within  this  category  all  involved  a lack  of 
knowledge  concerning  how  and  when  the  various  techniques, 
tools,  methodology,  and  procedures  should  be  used  ( 2 :50-5^. 
115-117) . 

Captains  Barga  and  Poch  in  their  technical  memorandum 
for  the  Air  Force  Aero-Propulsion  Laboratory  indicate  that 
during  the  conceptual  and  validation  phases  of  the 
acquisition  process,  there  is  often  a lack  of  adequate, 
complete  or  firm  system  definitions.  The  specifications, 
drawings  and  statements  of  work  are  not  available  in  detail, 
because  the  decision  of  which  alternative  design  to  use  has 
not  yet  been  made.  The  weapon  system  design  is  still 
undergoing  research  and  development.  Because  there  is  a 
lack  of  com.plete  data  and  only  historical  data  on  analogous 
systems  is  available  during  the  conceptual  and  validation 
phases,  a parametric  approach  to  ccst  estimating  should  be 
used  (3  !3i19)»  ^lore  information  becomes  available  during 
the  full  scale  development  and  production  phases.  Additional 
data  on  system  specification  and  performance  requirements 
become  available  as  the  weapon  syste.m  is  better  defined. 

Kore  identification  and  prediction  of  the  information  not 
available  is  also  possible  during  the  full  scale  developm.ent 
and  production  phases  ( 3 : 19-22).  For  this 


reason,  more 


accurate  methods  cf  estimating  weapon  systems  costs  can  be 
used.  A detailed  engineering  approach  becomes  possible 
during  the  full  scale  development  and  production  phases 
(3:19). 

The  amount  or  level  of  data  available  to  make  cost 
estimates,  as  described  by  Baker  and  Barga  (2:50-54)  and 
Poch  (3: 3 1 19-20),  can  be  placed  on  a continuum  ranging  from 
little  or  no  hard,  factual  data  available  to  a highly 
detailed  level  of  data.  For  the  purpose  of  this  research, 
the  researchers  have  defined  four  categories  of  daca  levels. 
These  levels  are  based  on  the  activities  taking  place  in  the 
phase  of  the  weapon  system  acquisition  process,  i.e.,  as 
the  system  progresses  through  the  acquisition  process,  it 
becomes  better  defined  and  moves  from  a paper  concept  to 
an  -actual  item  in  the  Air  Force  inventory.  The  four  levels 
of  data  as  defined  by  the  researchers  are: 

1.  VAGUE--In  the  conceptual  phase,  there  is  an 
almost  complete  absence  of  factual  cost  data  on  the  specific 
system  being  worked.  The  new  weapon  system  is  basically  a 
paper  system  and  has  not  been  defined  beyond  the  stage  of  a 
requirement  for  a particular  type  cf  weapon  system  to  meet  a 
need  (11:4-8;  12:23-29).  There  is,  however,  specific  data  on 
other  weapon  systems  available  for  use,  but  the  relation  to 
the  new  weapon  system  is  questionable  (3:3.  19). 

2.  LIMITED--In  the  validation  phase,  the  design  of 
the  system  begins  to  take  shape.  Mere  specific  information 


is  available  on  what  the  new  system  will  be  and  what;  the 


specifications  of  the  system  are  (12:23-29).  Because  this 

additional  inform.ation  is  available,  a betcer  comparison  to 

other,  previous  weapon  system.s  can  be  made,  giving  the  ^ 

I 

estimating  agencies  more,  but  still  limited,  data  on  which 
to  base  their  estimates  ( 3 : 19-22). 

3.  DETAILED--In  the  full  scale  developm.eni;  phase, 
the  system  is  completely  defined  and  the  specifications 
finalized.  Prototypes  of  the  weapon  system  are  developed 
for  testing  (12:23-29).  It  is  in  this  phase,  that  the 
majority  of  detailed  data  for  making  cost  estimates  becomes 
available.  Limited  production  figures  from  the  prototype 

production  are  also  available  for  use.  < 

2.  HISTORICAL-- In  the  production  phase  the  full 
production  figures  are  available  in  the  form  of  cost 
performance  reports  and  other  accounting  reports.  At  this 
point,  except  for  changes  in  the  production  schedule  and 
future  inflation  rates,  the  actual  cost  of  producing  the 
weapon  system  is  easily  and  accurately  projected  (ll  :L-8! 

12:  23-29;  25  ) . 

Techniques  Used  to  Generate 
Cost  Estimates 

From  a review  of  the  available  literature  it  is 
apparent  there  is  little  agreement  as  to  the  class  if icaticn , 
description  and  name  of  cost  estimating  techniques.  Phillip 
Cstwald  has  described  in  detail  eight  "preliminary  methods" 
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(conference,  comparison,  unorlered  ranking,  unit,  expected 
value,  computer  simulation,  probability,  and  ordinal  scale) 
and  five  "detailed  methods"  (factor,  power  law  and  sizing 
model,  standard  time  data,  cost-estimating  relationships  and 
marginal  analysis)  (20:167-21?).  Tom  Bond  has  described 
five  "basic  methods"  (list  price,  cost  as  a function  of  cost, 
cost  as  a function  of  performance,  item  analogy  and  expert 
opinion)  (5:123-7).  G.  A.  Batchelder  has  described  three 
"major  methods"  (industrial  engineering,  analogy,  and 
statistical)  and  mentions  that  one  military  source  states 
there  are  two  (s^Tithesis  and  analysis)  while  another  states 
there  are  four  (analytical  appraisal,  com.parative  analysis, 
statistical  analysis,  and  standards)  (4:l-io)  R.  S. 

Harrison  has  described  four  "basic  methods"  (empirical, 
comparative,  statistical  and  standards)  (l?:4-6).  An 
introductory  short  course  in  cost  estimating  and  analysis 
sponsored  by  the  Comptroller,  Air  Force  Armament  Development 
and  Test  Center,  Eglin  AFB , Florida,  mentions  nine  cost 
estimating  techniques  (parametric,  cost- to-cost , cost-to- 
noncost,  analogous,  engineering,  industrial  engineering, 
rates  and  factors,  simulation,  and  trend  analysis)  and 
describes  four  of  the  "major"  ones  (parametrics , analogous, 
engineering  and  trend  analysis)  (48:1.7-2.66).  The 
Department  of  DefenS'  through  the  Arm.ed  Services  Irccurement 
Reg'alation  Manual  (ASlM  No.  1)  describes  three  "mere  common 
methods"  (round  table,  comparison,  and  detailed)  (23:  2525). 


As  can  be  seen  there  are  many  different  tecaniaues 
used  in  the  derivation  of  cost  estimates.  The  choice  of 
which  technique(s)  to  use  in  any  given  situation  is  therefore 
subjective  and  confusing.  The  introductory  short  course 
Cost  Estimating  and  Analysis  gives  some  assistance  in 
making  the  choice  by  identifying  four  factors  to  be 

[ evaluated  before  making  the  choice.  These  items  are: 

*•*  1 . the  type  of  data  that  is  available  at  the  time 

the  estimate  is  being  made; 

2.  the  puroose  for  which  the  cost  estimate  is  to 

► 

I be  used: 

i 3*  the  accuracy  that  is  desired/required  of  the 

estimate;  and 

the  restrictions  of  time  and  resources  within 
which  the  estimate  is  to  be  made  (hS:2.73)* 

George  L.  Martin,  Executive  Secretary  of  the  National 
Estimating  Society  (NES),  in  a letter  to  the  Cost  Accounting 
Standards  Board  also  concludes  that  there  are  a large 
number  of  techniques  to  use  in  the  preraration  cf  cost 
estimates  and  that  the  situation  should  determine  the 
technique  to  use.  He  emphasizes  that  I«ir.  R.  S.  Harrison, 


1.  Define  the  work  to  be  done  in  as  much  detail  as 
possible  and  reasonable.  This  means  forecasting  the 
details  of  the  work  to  be  done  and  the  kinds  and 
quantities  of  miaterials,  parts,  and  ea.uipment  which  wil 
be  required, 

2.  Estim.ate  the  man  hours,  material  costs,  and 
other  cost-producing  elements  as  well  as  the  elapsed 
time  required  to  perform  each  detail  of  the  work, 

3.  Estimate  the  costing  rates  and  factors  for  the 
work  to  be  done, 

4.  Apply  the  costing  rates  and  factors  to  cost- 
prodvcing  elements  to  establish  total  plant  costs. 

5.  Evaluate  the  costs,  make  any  adjustments  to  the 
cost  estimate  that  are  required,  apply  the  desired 
profit  and  other  cost  factors  and  prorations  or 
additions,  and  establish  a sales  price. 

6.  Prepare  for  submittal  to  the  customer  [17:3--+'. 

These  steps  may  seem  simple  and  obvious,  but  they  are  basic 
and  worth  noting  by  any  cost  estim.ator.  No  matter  what 
technique  or  combination  of  techniques  is  used,  tne  steps 
are  still  valid  (15:^).  By  combini.ng  the  items  to  look  at 


and  the  steps  to  follow,  the  technique  to  use  should  be 
alm.cst  self-evident. 

For  ease  of  haniling,  the  researchers  have  broken 
the  cost  estimating  techniques  into  four  main  categories 
( Parametrics , Research  and  Development,  Engineering,  and 
Standards)  which  correspond  to  a given  phase  in  the 
acquisition  cycle;  Pax-ametrics  to  the  Conceptual  phase; 
Research  and  Development  to  the  Validation  phase; 
Engineering  to  the  Fixll  Scale  Development  phase;  and 
Standards  to  the  Production  phase.  This  classification  was 


selected  so  that  the  specific  technique  would  be  grouped  in 
terms  of  the  data  required  to  apply  the  technique.  These 
categories  correspond  to  the  data  available  in  the  various 
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phases  of  the  major  weapon  system  acoyaisition  cycle.  The 
techniques  have  been  placed  in  categories  consistent  with 
the  majority  of  the  literature  reviewed. 

Explanation  of  individual  cost  estimating  techniques. 
The  following  cost  estimating  techniques  are  explained  as 
they  will  be  used  throughout  this  research  effort. 

Techniques  were  combined  if  their  descriptions  and  degree  of 
input  data  were  similarly  explained  in  the  literature. 

Table  number  1 presents  a compilation  of  the  cost  estimating 
techniques.  This  table  shows  the  techniques  grouped  by 
category.  Because  of  the  widespread  use  of  different  names 
for  the  same  basic  technique,  the  additional  names 
associated  with  the  same  or  similarly  described  techniques 
have  been  added  for  the  readers  convience.  The  cost 
estimating  techniques  are: 

1.  Analogy  Cost  Estimates--This  method  is  derived 
by  choosing  analogous  programs  that  have  already  been 
completed  and  for  which  cost  data  is  available.  A ratio  cf 
the  degree  of  similarity  is  then  defined.  The  cost  estimate 
is  produced  by  multiplying  the  similarity  ratio  by  the  cost 
of  the  analogous  project.  .According  to  Bond,  this  is  the 
Ticst  comm.on  estimating  m.ethod  in  use  today  (5  :126).  It 
also  requires  a high  degree  of  knowledge  about  the  design 
and  mechaniza"!":  on  scheme  (E8:2.5S)  as  well  as  expertise  on 
the  part  of  the  estimator  (4  :7)*  Other  nam.es  for  this 

method  are  comparative  i 20;  26);  power  law  and  size 
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Statistical  (grass  roots,  expected  Engineering  type 

value,  probability,  and  with  gaps  filled 

ordinal  scale)  by  parametrics 


Standards  Standard  Time  Data  Historical  standard 

time/cost  by  task 


model,  where  only  the  size  of  the  item  changes  (20!l?l-4); 
factoring,  where  the  degree  of  each  similarity  is  called  a 
factor  (20:196):  unordered  ranking  (20:171);  and  unit  method 
(20  : 17^)  . 

2.  Computer  Simulation  Cost  Estimating--In  this 
technique , 

Simulation  is  defined  as  the  manipulation  and 
observation  of  a synthetic  m.odel  representative 
design  which,  for  technical  or  economic  reasons,  is 
not  susceptible  to  direct  experimentation.  This 
synthetic  model  ideally  represents  the  essential 
characteristics  of  the  real  system  with  the  frills 
excluded  [20:177], 

Once  the  simulation  is  programed  in  the  computer,  the 
"real  system"  data  is  entered  and  the  simulation  produces 
the  projected  cost  output. 

3.  Cost  Estimating  Relationships  (CER)--This  method 
includes  cost  as  a function  of  cost,  cost-to-cost , cost  as 

a function  of  performance  and  cosx-tc-noncost  techniques. 

It  uses  one  or  mors  independent  variables  which  could  be 
related  costs,  performance  characteristics,  etc.,  and 
derives  a cost  estimate  based  upon  historical  estimating 
relationships  from  a related  program.  Two  assumptions  must 
be  met  for  this  estimate  to  be  valid.  The  first  assumption 
is  that  the  new  program  will  be  affected  in  the  same  way  as 
the  original  by  the  independent  variable.  The  second  is 
that  the  acceptable  CER  has  been  established  and  verified 
( 5 :12i*-6;  13  = 1-3:  17  = 5:  20:207). 

E.  Engineering  Cost  Estimates --This  method  is  the 
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most  detailed  of  all  the  techniques.  It  involves  breaking 
down  the  project  into  detailed  work  segments  which  can  be 
individually  estimated  at  an  assumed  high  level  of  accuracy. 
These  estimates  are  then  summed  into  a total  cost  estimate 
(4:2;  48:2.60).  It  has  also  been  called  detailed  cost 
estimating  (28:2B28). 

5.  Expert  Opinion  Cost  Estimates--This  is  a purely 
subjective  technique  and  is  difficult  to  analyze  and 
substantiate.  The  expert's  opinion  must  be  accepted  as 
valid.  The  expert  must  completely  understand  all  of  the 
factors  in  the  problem.  This  method  is  best  applied  on  new 
products  which  are  beyond  the  technical  state  of  the  art 

( 5 : 126-7).  Another  name  for  this  method  is  empirical 
cost  estimating  (17  : 4)  . When  a group  of  experts  is  used, 
this  technique  is  called  the  conference  method  (4:168-9) 
or  round  table  estimating  (28:2B28). 

6.  List  Price  Cost  Estimate--This  method  requires 
that  the  items  to  be  estimated  have  a historical  price  that 
is  available  and  acceptable.  The  estimator  only  has  to 
multiply  the  units  required  by  the  accepted  price  and  the 
cost  estimate  is  complete.  This  is  the  most  accurate 
method  (5s 123)  given  that  accurate  list  prices  are  available. 

7.  Marginal  Analysis  Cost  Estimating- -This  type 
of  estimating  is  used  to  cost  out  changes  in  operations  or 
engineering  changes  for  a product.  The  change  usually 
involves  incremental  amounts  on  relatively  large  orders 
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The  primary  tool  used 


( i-  :207-217).  The  primary  tool  used  ir.  marginal  analysis  is 
differential  calculus,  a tool  which  is  beyond  the  scope  of 
this  study  to  describe.  An  excellent  explanation  can  be 
found  in  Cost  Estimating  For  Engineering  and  Management , a 
197^  publication  by  Phillip  F.  Cstwald  (20 : 207-217) . 

8.  Parametric  Cost  Sstimates--This  technique  is 
accomplished  by  correlating  design  parameters  to  historical 
costs  with  the  use  of  regression  analysis.  The  results 
describe  cost  relationships  between  the  parameters  (4S:2.2A), 
These  relationships  applied  to  the  desired  parameters  will 
result  in  the  estimated  cost. 


9.  Standard  Time  Data  Cost  Estimating — This 
technique  uses  historical  cost  data  to  find  the  standard 
cost  or  time  to  complete  a given  task.  Cnee  a set  of 
standards  is  known,  f'uture  event  schedule/cost  can  be 
calculated.  The  standard  for  a task  is  multiplied  by  the 
number  of  times  that  task  must  be  accomplished.  Then  all 
computed  values  are  summed  to  the  final  cost  estimate 
(17:6;  20  : 202-6) . 

10.  Statistical  Cost  Estimating--This  technique  is 
used  to  estimate  the  entire  job  by  using  major  parameters 
or  tech:iical  characteristics,  such  as  weight  or  speed,  to 
obtain  costs  for  major  portions  or  the  whole.  It  is 
similar  to  a macro  engineering  technique  with  parametrics 
added  to  fill  in  any  gaps  (17  O!  20:2-3).  "This  is 
sometimes  referred  to  as  'grass-roots'  estimating  [20:2'"." 


32 


ether  names  for  similar  techniques  are  expected  value 
(20:1?5).  probability  (20:181),  and  ordinal  scale  (2C:lS3). 


11.  Trend  Analysis  Cost  Zstimating--This  technique 
examines  the  cost  schedule  trend  patterns  during  the  full 
scale  development  and  production  phases.  These  trends  are 
then  used  to  project  the  anticipated  cost  to  complete  the 
contract  (l-S  ; 65 ) . 

Categories  of  cost  estimating  techniques.  As 
discussed  earlier,  the  amount  and  detail  of  data  increases 
across  the  weapon  system  acquisition  cycle.  For  this  reason, 
the  following  categories  are  based  on  the  amount  and  detail 
of  data  available  at  the  corresponding  phase  in  che  weapon 
system  aco_uisition  process. 

It  should  be  noted  that  the  techniques  used  in  the 
earlier  phases  can  be  used  throughout  the  weapon  system 
acquisition  process.  As  the  amount  of  available  data 
increases,  other  techniques  become  usable  that  provide 
increased  accuracy.  For  these  reasons  the  techniques  appear 
in  those  categories  where  they  will  produce  their  greatest 
accuracy.  The  cost  estimating  technique  categories  are: 

1.  Parametric  Category--In  this  category  the  cost 
estimates  come  from  relationships  which  can  be  developed 
between  historical  costs,  system  physical  attributes  and/or 
performance  characteristics.  The  historical  costs  take  into 
account  system  growth,  engineering  changes,  program. 
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stretch-outs  and  any  other  possible  difficulties  encountered 
in  comparable  programs  (13:5.^:  They  have  valuable 

application  when: 

a.  Some  performance/design  parameters  are  known 
but  detailed  mechanization  features  are  lacking. 

b.  Gross  estimates  are  acceptable. 

c.  In  the  early  stages  of  program  development. 

d.  Quick  reaction  estim.ates  are  needed. 

e.  Used  for  cost/performance  trade-off  studies 

(h8:2.50) . 

The  cost  estimating  techniques  which  fit  into  this 
category  include  analogy,  CER,  expert  opinion,  list  price, 
and  parametric. 

2.  Research  and  Development  (R&D)  Category--In  this 
category  the  cost  estimates  are  developed  by  a combination 
of  the  parametric  and  engineering  techniques.  Data  on 
critical  portions  cf  the  project  is  generated  through 
engineering  test  and  research.  This  data  is  then  used  with 
the  engineering  technique  to  develop  the  costs  cf  these 
critical  aspects  of  the  program.  Those  parts  of  the  project 
that  are  left  are  estimated  using  the  parametric  techniques. 
The  resulting  cost  esti.mate  therefore  falls  in  the  middle 
as  far  as  detail,  time  to  complete  and  position  in  the 
major  weapon  system  acquisition  process  are  concerned.  Its 
major  contribution  is  that  of  an  improved  and  updated  cost 
estimate  for  review  at  BSARC  II  (14). 
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The  cost  estirnaxing  xechniques  which  fit  into  this 
caxegory  include  analogy,  simulation,  CER , engineering, 
expert  opinion,  list  price,  marginal  analysis,  paramexric  and 
statistical. 

3.  Engineering  Category--In  this  category  the  cost 
estimates  are  made  by 

. . . defining  the  effort  to  be  accomplished,  the 
schedule  it  is  to  be  accomplished  against,  and  xhe 
maxerials  and  processes  to  be  utilized;  parceling 
xhis  information  out  to  the  performing  organizations 
who  estimate  the  cost  for  each  work  package  i_48:2.60]. 

They  have  valuable  applicaxions  when: 

a.  Detailed  data  is  available  and  detailed 
esximates  are  desired. 

b.  Contracxors  are  calculaxing  bid  prices. 

c.  Time  is  available  to  properly  complete  the 
process.  (This  process  is  time  consuming)  (2-8:2.68). 

The  cost  estimating  xechniques  which  fix  inxo  this 
category  include  sim.ulation,  engineering,  marginal  analysis, 
standard  time  data,  statistical  and  trend  analysis. 

2-.  Standards  Category--In  this  category  the  cost 
estimates  are  developed  from  historical  cost  data. 

Standard  esti.mates  are  developed  by  studing  xhis  daxa  base 
and  conducting  time  and  motion  studies  on  combinaxions  of 
xhese  costs  (17:6).  Other  statistical  methods  such  as 
regression  analysis  can  be  applied  to  this  historical  data 
to  yield  standards  which  can  be  used  to  predict  future 
costs  (20:203).  Some  of  the  advantages  of  the  standard 
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estimates  are: 

a.  The  data  is  cheaper  and  easier  to  gather. 

b.  The  estimates  are  more  consistent. 

c.  It  is  easier  to  understand  its  deviation 

(20 : 203)  . 

The  COST  estimating  techniques  which  fit  into  this 
category  include  standard  time  data  and  trend  analysis. 

3 umm.ary 

Figure  2,  Model  of  the  Air  Force  Cost  Estimating 
Process  :Vithin  the  Air  Force  Weapon  System  Acquisition 
Process,  summarises  the  literature  review.  The  Air  Force 
weapon  system  acquisision  is  used  as  she  time  frame  upon 
which  the  model  is  built.  The  agencies  identified  as 
making  estimates,  the  type  of  estimate  they  make,  the  type 
information  available  to  them  to  make  the  estimates  and 
the  techniques  used  in  making  the  estimate  are  presented. 
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The  contractor  & SrC  oan  make  more  than  one  estimate  duri: 
each  phase.  An  estimate  is  made  and  revised  until  time  of 
oresentation  to  the  DSARC'  for  a decision. 


The  Si"0  & Contractor  personnel  make  continual  on-going 
cost  estimates  throughout  the  production  phase. 


CHAPTER  III 

j<2:thcdclggy 

Description  of  Universe 
and  Population 

?or  this  research,  the  universe  is  defined  as  the 
set  of  all  agencies  which  make  cost  estimates  on  DCD  weapon 
systems  acquisitions.  The  population  is  defined  as  the  set 
of  all  DCD  agencies  which  make  cost  estimates  on  ma’or  Air 
Force  weapon  system  acquisitions.  This  includes  the  CSD 
CAIG,  Major  Commands,  and  System  Program  Offices. 

Description  of  the  Sample 

This  research  was  conducted  using  a selected  sample 
consisting  of  the  OSD  CAIGj  the  Comptroller,  AFSi  and  AFLC ; 
available  SPCs  from  ASD,  SSD,  and  SAIvSC ; and  selected  Major 
Commands  presently  submitting  estimates  on  major  weapon 
systems  currently  in  the  acquisition  process.  At  least  three 
individual  estimators  (when  pcssible)  from  each  agency  will 
be  interviewed  to  obtain  the  organizational  view  and  to 
reduce  the  problem  of  individual  bias.  The  sample  v/as 
selected  for  it  perceived  representativeness  relative  to  the 
population  and  for  the  remainder  of  this  study  it  is 
assumed  that  ■^he  sample  can  be  considerec  representative  of 
the  population. 


The  rationale  for  this  assumption  is  as  follows: 

1.  The  OSD  CAIG  is  ~he  same  agency  in  toth  the 
sample  and  the  population  and  therefore,  represents  a 
census  for  that  portion  of  the  population. 

2.  A!^SC  and  AFLC , as  applicable,  are  generally 
selected  as  the  major  command  to  perform,  the  IC2  and 
prepare  the  Budget  Cost  Information  on  each  published  HCC. 
One  of  these  two  commands  will  be  responsible  for 
designating  the  office  of  pri.mary  responsibility  (C?R)  for 
each  ICE  and  t.he  team  chief  for  the  team  performing  the 
ICE.  Cne  of  these  two  commands  will  also  be  responsible 
for  reviev/ing  all  published  RCCs  and  providing  cost 
estimates  for  those  -RCCs.  The  cost  estimate  provided  are 
used  in  the  evaluation  of  proposed  alternatives.  Therefore, 
inclusion  of  the  Comptroller,  AESC  and  AFLC  as  part  cf  the 
samiple  should  adequately  represent  the  timing  and  techniques 
used  by  miajcr  command  independent  cost  estim.atcrs.  It  will 
also  represent  the  tim.ing  and  tecnniques  used  by  m.ajcr 
comjnands  to  provide  cost  estim.ates  on  the  RCCs  proposed  by 
agencies  which  identify  requirem,ents  for  new  weapon  systems. 

3.  The  AFSC  S?Cs  selected  for  inclusion  in  the 
sample  were  based  on  convenience  and  availability.  All 
current  .ASD  S?Cs  associated  with  m.ajor  weapon  systems  and 
a number  cf  3?0s  from  ESD  and  SATt^C  were  selected.  This 
permitted  the  researchers  to  sample  aeronautical  syste.ms 
including  aircraft  and  air  launched  missile  weapon 
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systems  (ASD) , major  electronic  systems  (^SD) , and  ballistic 
missile  and  space  systems  (SAIviSC).  A representative  sam.ple 
of  jhe  population  'vas  thus  produced. 

By  having  a census  of  ASD  SFGs,  the  researchers 
identified  the  techniques  used  and  amount  of  data  available 
within  one  portion  of  AFSC  for  making  official  program 
estimates.  Smialler  selections  of  BSD  and  SAJiSC  SFCs  v/ere 
selected  to  verify  that  the  same  techniques  and  data 
availability  identified  within  ASD  were  representative  of 
AFSC. 


Data  Collection 

The  .model  of  the  Air  Force  .major  weapon  system 
acquisition  process  was  developed  by  a detailed  review  cf 
the  available  literature,  including  .AF  and  DCD  regulations, 
manuals,  pamphlets,  directives,  and  ether  publications. 

The  literature  review  identified  the  weapon  system. 


acquisition 

process 

phases,  ' 

the  agencies 

involved  in 

miaking  cost 

estim.at 

es  during 

the  process 

(the  sources  o 

the  cost  es 

oimates ) 

, and  the 

variety  of 

techniques  used 

m.ake  the  cost  estimates.  The  literature  review  was 
conducted  in  response  to  research  question  number  1 . 

The  model  constructed  during  the  literature  review 
was  validated  using  e.xtensive  unstructured  interviews  with 
individuals  from  each  of  the  CCD  agencies  identified  in  the 
miodel.  While  the  unstructured  personal  interview  was  the 
method  of  choice  in  this  study,  it  was  not  always  possible 


o personally  meet  with  each  interviewee. 


The  personal 


interview  was  therefore  supplemental  as  reo_uired  with 
telephone  interviews. 

Before  begiruiing  the  data  collection  effort,  an 
"Interview  Schedule"  was  prepared  as  a giuide  for  the 
unstructured  personal  interviews.  A sample  of  this  schedule 
is  presented  in  Appendix  A.  The  interview  sc.hedule  consists 
of  a line  of  sub-questions  developed  to  answer  research 
question  number  2.  The  interview  schedule  was  sufficiently 
structured  to  insure  coverage  of  the  research  q_uesticn,  but 
not  so  rigidly  structured  as  to  preclude  the  flexible 
probing,  checking,  and  cross  checking  of  data.  Detailed 
notes  were  taken  during  each  interview.  Ilany  interviews 
were  also  recorded  on  a small  portable  tape  recorder  for 
later  more  detailed  analysis. 

Before  conducting  the  telephone  interviev/s , contact 
was  made  with  each  interviewee  to  establish  the  date/tim.e 
for  the  formal  telephone  interview.  A copy  of  the  Interview 
Schedule  was  sent  to  each  interviewee  prior  to  the  actual 
interview.  During  the  telephone  interview  detailed  notes 
were  taken  and  expanded  immediately  after  the  interview 
session.  liany  telephone  interviews  were  also  tape  recorded 
for  later  detailed  analysis. 

The  time  available  to  conduct  the  telephone 


interviews  was  frequently  limited.  Because  cf  this,  it  was 
not  always  possible  to  cover  the  entire  Interview  Schedule 


in  detail.  Therefore,  the  sections  on  when  estimates  are 
made,  amount  of  data  available,  and  techniques  used  were 
stressed  in  every  interview. 

These  unstructured  interviews  were  designed  to 
ascertain  the  following  information: 

1.  Name  of  individual  interviewed--all  individuals 
interviewed  were  practicing  estimators,  currently  involved  in 
the  estimating  process.  As  such,  they  provided  expert 
opinion  and  data  for  t.he  study.  However,  to  insure  candor, 
the  individuals  were  offered  a guarantee  that,  if  they 
desired,  their  names  would  be  held  in  strict  confidence, 

2.  Organization  to  which  the  individual  belongs-- 
the  organization  of  the  individual  was  ascertained  to 
demonstrate  that  the  sample  included  a wide  representation 
of  the  population. 

3.  Length  of  time  the  individual  has  been  in 
current  pos ition--this  data  was  necessary  to  establish 
the  expert  status  of  the  individual  as  a practicing 
estimator  representing  the  views  of  his  organization. 

d.  Individuals  baclcground  in  esti.mating--this  data 
was  also  necessary  to  establish  the  expert  status  of  the 
individuals  as  a practicing  estimator. 

5.  Details  of  the  estimating  techniques  the 
individual  uses  in  any  phase  of  the  weapon  system  acquisition 
process  where  the  individual  makes  estimates--this  data  was 
necessary  to  validate  the  model  developed  in  Chapter  II 
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and  insure  that  the  categories  developed  are  accurate. 

6.  The  individual's  views  concerning  the  usefulr.es 
of  the  techniques  used--this  data  was  used  to  provide  a 
better  insight  into  the  different  estimating  techniques  and 
to  possibly  improve  the  model  and  the  estimating  process. 

7.  The  individual's  views  concerning  whether  the 
techniques  used  are  the  most  appropriate  techniques 
available-- this  data  was  used  to  modify  the  model,  if 
appropriate,  and  to  provide  insight  into  different  ways  to 
possibly  improve  the  estimating  process. 

Criteria  for  Research 
and  Gomoarison 

The  validation  process  involved  interviewing  large 
numbers  of  people  from  organisations  that  make  cost 
estimates  to  validate  the  information  obtained  from  the 
literature  review.  It  involved  the  subjective  judge.ment 
of  the  researcher,  but  this  judgement  was  based  on  detailed 
analysis  of  all  information  available. 

The  criteria  established  for  this  research  was  that 
for  the  model  to  be  valid,  the  real  world  estimating 
environment  would  have  to  match  the  model  ICO  per  cent  of 
the  time  for  the  following: 

1.  An  agenoy  will  make  an  estimate  during  each  of 
the  acquisition  phases  as  specified  in  the  model  for  that 
particular  agency. 

2.  The  technique  actually  used  by  the  agency 
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matches  the  technique  category  specified  in  the  model  for 
that  agency  in  that  phase  cf  the  acquisition  process. 

Assumotions  and  Limitations 


In  developing  and  validating  the  model,  it  will  be 
necessary  to  accept  certain  limitations  and  to  m.ake  certain 
assumptions.  These  limitations  and  assu.mptions  are 
considered  to  be  an  integral  part  of  the  research  design. 

Assum-pticns . The  following  assumptions  applied  to 
this  research  effort: 

1.  The  data  obtained  from  individual  estimators 
through  the  unstructured  interviews  was  unbiased. 

2.  The  technique  categories  identified  in  the 
model  include  all  techniques  available  to  the  estimators. 

3.  The  available  techniques  are  properly  cate- 
gorized . 

The  sample  can  be  considered  representative  cf 
the  population. 

5.  A census  of  ASD  SFCs  and  a smaller  sample  cf 
ESD  and  SAMSO  SFGs  was  considered  representative  of  the 
techniques  used  and  amount  of  data  available  within  all 
AFSC  SPCs  for  making  Official  Program  Estimates. 

6.  The  amount  of  data  available  and  the  techniques 
used  by  the  civilian  contractors  is  dictated  by  the  SPC 
associated  v/ith  the  particular  weapon  system. 
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Limitations . The  following  limitations  applied  to 
r*9S03.2TC}l  Q^To^'Zi 

1.  The  data  collection  is  limited  by  the  ability 
of  the  authors  to  colleot  the  reo.uired  data  and  by  the 
ability  and  willingness  of  the  data  source  to  provide  the 
reo^uired  data. 

2.  A number  of  interviews  had  to  be  conducted  by 
telephone  rather  than  in  person  due  to  time  and  travel 
limitations . 

3.  The  validity  of  the  study  was  not  reduced 
because  of  the  sampling  technique  used  in  the  AFSC  S?Cs. 

h.  Only  LCD  agencies  were  considered  in  this 
research  effort.  Civilian  contractors  were  considered  an 
uncontrollable  factor  and  were  thus  eliminated  from  the 
population. 
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CHAPTER  IV 
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I 

I 

ANALYSIS  AND  SULCVIARY  OF  FINDINGS 


Approximately  30  individual  estimators  from  all  the 
organisations/agencies  involved  in  estima-^ing  the  costs  for 
the  acquisition  of  new  AF  weapon  system  were  interviewed 
as  part  of  this  study  as  a validation  effort  for  the  model 
presented  in  Chapter  II.  This  chapter  is  an  analysis  of  the 
interviews  by  organization/agency . Each  organisation/agency 
is  looked  at  in  detail  to  ascertain  when  they  make  estimates, 
the  techniques  they  use  and  why  the  estimate  is  made.  A 
comparison  of  the  literature  model  and  the  real  world  model 
v/ill  be  made.  The  chapter  will  be  concluded  wizh  some 
general  impressions  which  cut  across  all  organizations/ 
agencies.  The  development  of  the  real  world  model  and  the 
impressions  are  based  upon  the  interviews  conducted.  The 
impressions  form  the  basis  for  many  of  the  conclusions 
reached  in  Chapter  V. 

The  Operational  PAJCCMs  and  the  AF  CAIG  were 
identified  within  the  course  of  the  data  collection  as 
potential  sources  of  cost  estimates  and  were  added  to  the 
research  effort.  Data  were  collected  from  these  organizations 
using  the  same  methodology'  outlined  in  Chapter  III  for 
collecting  data  from  the  previously  identified  organizations. 
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As  anticipated  in  the  initial  model,  the  individual 
estimators  in  all  organizations/agencies  visited  perceived 
that  the  amount  of  data  available  for  use  in  generating  ccsz 
estimates  grows  in  quality  and  quantity  as  the  weapon  system 
progresses  reward  turn-over  to  the  using  coimmand.  -hey  also 
perceived  the  data  availability  to  be  closely  cor-nected  to 
the  phase  of  the  acquisition  cycle  that  the  program  was  in 
and  the  degree  to  which  the  program  was  defined. 

During  the  conceptual  phase,  the  estimarors  work 
primarily  with  concepts  developed  to  satisfy  the  RCCs,  The 
data  is  based  upon  paper  requirements.  There  is  little  scope 
or  content  to  the  data,  which  is  considered  scarce  and  hard 
to  obtain.  Estimates  are  based  on  the  "educared  g'oesses"  of 
the  engineers  and  m.anagers  associated  wirh  the  program.  As 
the  program  moves  into  the  validation  phase,  the  number  of 
different  concepts  under  consideration  is  narrowed  dcvcc. 

The  weapon  system  is  further  defined,  allowing  the  estimator 
to  make  analogies  to  other  programs  with  more  confidence. 

The  amount  of  data  continues  to  grow  in  the  full  scale 
develcpmient  phase  as  the  weapon  concept  is  narrowed  dovm  to 
one  system  and  the  contractor  data  from  prototype  production 
becomes  available.  In  the  production  phase,  the  estimator 
has  a much  broader  data  base  to  work  v/ith.  Actual 
production  data  becomes  available  on  the  weapon  system. 

Also,  the  Research  and  Development  (R&D)  effort  on  the 
weaoon  system  is  com.oleted  and  the  costs  for  it  are  known. 
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This  eliminates  estimating  the  most  difficult  (high  risk) 
portion  of  the  weapon  system  acquisition,  cut  in  most  cases 
"estimates"  are  no  longer  needed  as  a basis  for  obtaining 
funds  from  Congress  since  these  f'jnds  have  already  been 
received  and  committed. 

The  amount  of  data  available  to  the  cost  estiimators 
matches  the  m.odel  for  all  organizations/agencies  for  the 
phases  in  which  they  generate  cost  estimates.  The  individual 
cost  estimiators  all  reported  using  the  data  that  they 
perceived  as  the  best  available  at  the  time  to  generate  a 
cost  estimate. 

The  data  grov/th  was  the  orTLy  area  of  the  model  that 
resulted  in  iCOfo  agreement  across  all  organisaticns/agenc ies . 
The  remainder  of  this  chapter  will  present  the  data  on  each 
organization/agency  and  is  in  support  of  the  model  presented 
in  figure  1- (page  33). 

ArSC  and  AFLC  Recuired 
Coerational  Caca- 
bilitv"(RCC)  Cost 
Estimates 

HQs  AFLC  and  AFSC  are  tasked  to  provide  preliminary 
cost  estimates  for  new  RCCs.  AFLC  provides  the  preliminary 
cost  estim.ates  for  RCCs  that  can  be  satisfied  by  m.odifying 
the  conf igiuraticn  of  a current  item.  AFSC  provides  the 
prelim.inary  cost  estimate  for  RCCs  requiring  a research 
and/or  development  effort.  Both  headquarters'  monitor  and 
evaluate  the  RCC  cost  estimates  generated  under  their 
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HQ  AFLC  is  tasked  to 


direction  (39:  . 

AFLC  (RCC)  cost  estimates . 
provide  preliminary  cost  estimates  on  RCCs  that  can  be 
satisfied  by  modification  to  an  existing  weapon  system. 

This  cost  estimate  is  required  early  in  the  conceptual  phase, 
ihe  Flans  Division  (AFLC/XRX)  monitors  the  RCCs  and  the 
appropriate  Air  Logistics  Center  (ALC)  provides  all  cest 
estimates  on  the  proposed  modification.  ■’’These  cost 
estimates  are  called  Modification  Proposal  Analyses  (‘■-PA) 

(39).  The  data  available  this  early  in  the  acquisition 
cycle  is  very  limited.  There  is  generally  little  or  no 
program,  cost  data  and  only  broad,  conceptual  ideas  (possible 
solutions)  to  satisfy  the  RCC.  In  some  cases  there  may  be 
analogous  programs  from  which  partial  data  is  available 
( be  : bl ) . 

The  pri.mary  techccique  used  to  generate  the  initial 
cost  estimate  is  that  of  round  table  discussions  by  .-XC 
technicians  and  logisticians.  If  any  data  is  available, 
from  the  contractor  on  the  original  configuration  item 
(weapon  system)  which  is  to  be  modified,  it  is  analysed  and 
modified  as  appropriate  to  fit  the  proposal  (40;  4i ) . 

These  technicians  and  logisticians  make  the  cost 
estimates  based  on  all  available  data  and  on  their  experience 
v/ith  other  progra.ms.  It  is  perceived  that  these  cost 
estimates  are  rough,  best  g’uesses  and  that  the  final  cost 
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nay  be  entirely  differen”  (b-C;  di ) . 

After  the  esrinare  is  submitted  by  the  ALC , it  can 
be  used  for  a comparison  with  other  possible  solutions  or 
in  evaluating  program  affordability.  These  cost  esti.mates 
are  not  used  for  specific  funding  purposes,  but  only  as  a 
basis  for  trade-cff  analyses  (39) • 


AF3C  (RCC)  cost  estimates.  HQ  AFSG  is  tasked  to 
provide  preliminary  ccst  estimates  on  RCCs  rec.uiring  a 
research  or  development  effort.  This  rsouirement  occurs 
in  the  early  conceptual  phase  cf  the  acquisition  cycle  for 
the  proposed  program.  The  Policy  and  Programs  division 
(.nFSC/X?.X)  monitors  the  RCCs  and  the  Cost  Analysis  (AFSC/ 
ACCA)  and  Product  Divisions  (AAD/.ICC,  23 D/AC C , 3,i:.5C/.lCC  , 
A.DTC/ACC)  provide  support  i.n  preparing  the  cost  estimates. 
These  are  not  the  cost  estimates  used  for  BS.IRC  I,  but 
m.erely  early  rough  g’uesses  (Db). 

There  is  virtually  no  program-related  co; 
data  available  at  this  point  in  the  acquisition  process. 

The  only  inform.ation  available  is  the  statement  of  an 
operational  requirement/capability  that  does  net  presently 
exist.  To  solve  this  lack  of  data  problem,  command  experts 
are  asked  to  provide  possible  techuiical  solutions.  For 
eaoh  alternate  solution  an  attempt  is  made  to  "develop 
production  support  costs  and  total  life  cycle  ccs~  estimates 
(to  include  risk  assessm.ent  for  each  alternative^ 


estim.atina 


These  cost  estimates  are  generated  using  the  round  table  and 
expert  opinion  techniques.  The  cost  estimates  Tor  each 
proposed  technical  solution  are  made  using  the  same  criteria 
and  methodology.  This  is  necessary  so  that  the  estimates  car 
be  readily  compared  with  each  other  and  a rough  ccst  analysis 
or  trade-oTT  between  the  prooosed  solutions  made  (~b). 

The  HQ  AFSC  estimators  believe  that  the  cost 
estimates  generated  this  early  in  the  aco^uisition  process 
are  "for  concepts  only  and  are  not  intende-  to  recresent 


exact  naroware  cos' 


'4  . 


inesr 


esti.mates  are  not 


user  for  specific  funding  purposes.  They  are  used  fcr 
comparison  of  technical  solutions  only  and  the  "estimates" 
are  rightfully  ignored  for  funding  purposes  by  the  field 


(■ 
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Summanv . HQs  AFSC  and  .AFLC  match  the  model 
coxipletely.  They  evaluate  ccst  estimates  cf  concepts  fcr 
satisfying  RCCs  in  the  early  part  of  the  ccncepfu?,!  phase. 
There  is  very  little  data  available  on  the  ne'w  concepts  at 
this  point  in  the  acquisition  process,  which  necessitates 
t.he  use  of  parametric  cost  estimating  techniques.  These 
cost  esti.mates  are  for  comparison  of  alternatives  only  and 
are  not  used  as  official  program  esti.mates  by  the  TSARC  or 
for  future  funding  purposes  (39;  ^4). 
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not  make  estimates  on  the  RCCs  they  generate  (5^;  55:  56:  57). 
Cnee  the  need  or  requirement  for  a new  weapon  system  to  meet 
an  operational  m.ission  is  generated,  the  ROC  is  forwarded 
to  -A.FLC  and/or  AFSC.  AFLC/AF3C  provide  the  concepts  or 
solutions  which  will  he  used  to  meet  the  rea_uirements  , as 
well  as  the  initial  cost  esti.mate  for  the  RCC.  The 
operational  MAJCCMs  rely  almost  motally  upon  AFL-C/AFSC 
generated  cost  esmimates  for  new  weapon  systems  55:  56: 

57). 

The  operational  MAJCOKs  do  not  make  "system 
acquisition"  cost  estimates  on  the  new  weapon  system  as  it 
progresses  from  a RCC  to  an  operational  system.  The  llAJCClis 
have  no  expertise  in  the  area  of  .Research  and  Development 
(Rd:D)  cost  estimating.  They  are  prim.arily  concerned  with 
tie  operations  and  support  (O&S)  or  operations  and 
maintenance  (CdiVI)  costs  rather  than  the  R<1D  costs  (5^:  55: 

56:  57). 

The  operational  IvIAJCOKs  become  involved  in  the 
acquisition  process  when  the  SPCs  recommend  requirem.ents 
tradeoffs.  The  primary  emphasis  of  operational  units  in 
this  area  is  to  limit  the  impact  cf  requirem.ents  tradeeffs 
provided  by  the  S?0s  v/hich  result  from  changes  in  technology 
and  may  reduce  the  capability  of  the  new  system.  Once 
again  though,  the  Fj\JCC.Vis  do  not  attem.pt  to  make  a system, 
cost  estimate.  Instead,  they  serve  as  a source  of  infor- 
m.ation  on  C<3:?/I  or  GIF  requirem.ents  and  costs  for  the  weapon 
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system  resulting  from  the  system  configuration  v/hich  '.vould 
result  from  the  alternatives  proposed  (5^:  55:  56;  5'^). 

System  Program  Offices 

Cost  estimates  are  generated  by  System  Program.  Offices 
of  three  different  AFSC  organisations.  Aeronautical  System.s 
Division  (ASD)  miakes  estimates  cn  aeronautical  sy3tem.s-- 
aircraft  and  air  launched  weapon  3ystem.s,  Electronics 
Systems  Division  (SSD)  makes  cost  estimates  on  all  m.a,]cr 
electronic  weapon  systems.  Space  and  kissile  System:S 
Organisation  (SAf.SC)  makes  estim.ates  on  ballistic  missile 
and  space  systems. 

Aeronautical  System.s  Division.  ASD  SrCs  generate 
cost  estimates  in  all  four  phases  of  the  weapon  system, 
acquisition  cycle.  Within  ASD,  the  Directorate  of  Cost 
Analysis,  -Advanced  Systems  Division  (ASD/.ACCZ)  is  responsible 
for  generating  the  cost  estimiates  in  the  conceptual  phase. 

The  S?0s,  which  are  formed  toward  the  end  of  the  conceptual 
phase,  use  the  estimates  generated  by  -ASD/ACOX  as  their  cv.r. 
estimate  for  that  phase.  The  SFC  personnel  generate  their 
own  esti.mntes  in  the  remaining  phases.  "ASD/ACOX  dees  the 
initial  phase  estimating  [A-7]."  Therefore,  ASD/.ACCX  is 
considered  by  .ASD  personnel  to  be  t.he  SPG  for  the  major 
portion  of  the  conceptual  phase.  They  work  on  pre-.ECC 
studies  and  conduct  studies  to  meet  the  ROC  after  it  is 
developed  (dy) , 
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The  amount  of  data  available  for  use  by  the 
estimators  influenced  rhe  type  of  technicue  used  to  a great 
extent.  Personnel  from  one  S?C  emphasized  this,  claiming 
"The  technicue  used  is  driven  by  che  amount  and  type  of 
data  available  [47'."  The  ASD  5?C  estim.ators  use  parametric 
estimating  technio^ues  in  the  conceptual  and  validation 
phases.  The  RCA  PRICE  model  is  used  for  estimating  electronic 
eo.uipment.  CERs , expert  opinion,  and  analogies  are  relied 
upon  o_uite  heavily  for  generating  cost  estimates  in  both  of 
these  phases.  An  attempt  is  made  to  correlate  design 
oarameters  to  historical  cost  data  from  other  rros'ra.ms  and 


arrive  at  the  estimated  cost 


:ne  new  v/eaocn  system.  xn 


the  full  scale  development  phase,  the  estimators  use  scmiS 


params 


:ec.tniques,  ou" 


reiy  on  eng; 


techniques.  They  are  able  to  use  detailed  engineering, 
statistical  vgrass  roots),  marginal  analysis  and  trend 
analysis  techniques.  Prototype  v/eapon  systems  have  been 
developed  and  produced,  yeilding  a much  greater  data  base  to 
work  with.  The  estimators  also  have  the  v/ork  break  down 
oackaaes  from  the  Cost/Schedule  Control  System.s  Criteria 
(C/5CSC)  available  for  the  weapon  system  they  are  working  on. 
This  added  data  base  allows  the  estimator  to  use  the 
engineering  techniques.  In  the  production  phase,  actual 
production  data  is  available.  Historical  standard  time/cost 
by  task  and  historical  trends  are  used  to  generate  the  cost 
estimates  during  this  phase  (47). 


Cost 


estimates  are  generated  by  the  A£D  3?Cs  for  a 
variety  of  reasons,  the  primary  ones  being  for  program 
advocacy  to  the  A?SARC  and  DSARC  and  for  budget  preparation. 
Cost  estimates  are  also  used  for  program  control  and 
adjustment,  "what  if"  contingency  reviews,  and  in  support  of 
source  selection  and  contract  negotiations , In  the 
conceptual  and  validation  phases,  the  cost  estimates  are 
used  in  malting  comparisons  between  the  different  v/eapcn 
system.s  under  consideration.  Ihe  cost  estim.ates  give  the 
decision  makers  at  the  .Air  Staff  and  SCD  a gross  idea  of  the 
costs  involved  to  achieve  a certain  capability  (S?). 

In  summary,  t.he  .ASD  S?Gs  validate  t.he  model  in  all 
areas  except  technicue  used.  The  ASD  5?Cs  make  cost 
estim.ates  in  the  conceptual,  validation,  full  scale 
development,  and  production  phases  of  the  weapon  system, 
acquisition  cycle.  The  amount  of  data  available  for 
ge.nerati.ng  cost  estimates  grows  from  very  little  (vag’ue)  to 
extensive,  production  data  (historical).  The  quality  and 
quantity  of  data  grows  as  the  weapon  system,  progresses 
through  the  cycle  while  the  risk  and  the  tim.e  ever  which 
estimates  are  required  to  be  valid  aecreases.  The  3?Cs 


make  estimates  for  two  m.ajor  reasor.s--program.  advocacy  and 
budget  generation.  In  the  area  of  techniques  used,  the  .A3D 
SrCs  use  parametric  estim.ating  technioyues  to  generate  cost 


technicues  in 


full  scale  development  phase,  and  standards 
the  production  phase.  The  type  of  technique  used  is 
dependent  upon  the  amount  of  data  available.  The  individual 
estimator  choses  v/hich  technique  he  will  use  based  upon  his 
interpretation  of  the  amount  of  data  available  (1-7). 

Electronics  Systems  Division.  The  ESD  S?0s  make  cos 
estimates  in  the  four  phases  of  the  major  weapon  systems 
acquisition  process  (conceptual,  validation,  full  scale 
development  and  production).  The  only  case  where  the  S?C 
might  not  make  an  estimate  is  during  the  conceptual  phase, 
and  then  only  if  that  S?C  had  not  yet  been  formed.  From 
the  time  of  their  conception  until  their  dissolution,  the 
ESD  SPOs  maintain  a current  cost  estimate  fcr  their  program. 
This  estimate  is  updated  throughout  the  acquisition  cycle 
as  new  data  becomes  available  and  as  changes  are  m.ade  to  the 
program.  In  the  production  phase  the  basic  cost  estim.ate  is 
analyzed  to  get  estimates  of  time,  num.ber  and  cost  changes 
(50). 

All  ESD  SPC  estimators  perceive  that  the  techniques 
used  in  the  different  phases  of  the  acquisition  cycle  for 
making  cost  estimates  are  directly  related  to  the  amount  and 
type  of  data  available.  In  the  conceptual  phase,  the  RCA 
PRICE  model  and/or  a round  table  type  technique  are  used 
to  make  cost  estimates.  The  RCA  PRICE  m.cdel  uses  both 
objectively  and  subjectively  based  data.  The  round  table 


technique  consists  of  cost  analysts  and  i?C  engineers  working 
together  in  conference  to  develop  the  cost  estimate.  During 
the  validation  phase,  price  lists  (v/here  available)  and 
contractor  estimates  (adjusted  as  necessary  by  the  SPG)  are 
used.  Parametric  techniques  are  also  used  as  a check  on 
the  contractor  estimates.  In  the  full  scale  development 
phase  contractor  data  (again  adjusted  as  appropriate)  and 
parametric  techniques  are  used.  Finally,  in  the  production 
phase  the  contractor  estimates,  with  only  minor  adjustments, 
are  used.  The  ESD  SPC  estimators  believe  that  the  contractors 
use  parametric  techniques  and  price  lists  in  the  validation 
phase,  technical/engineering  techniques  in  the  full  scale 
development  phase  and  standards  (gained  from  actual 
production)  techniques  in  the  production  phase.  The  SPC 
makes  a check  for  reasonableness  of  the  contractors  estimate. 
This  check  is  made  by  the  technical/engineering  analysis 
personnel  using  parametric  techniques.  The  most  popular 
parametric  technique  used  is  the  RCA  PRICE  model  (qC). 

Cost  estimates  are  made  yearly  fcr  the  PCIi  and 
budgetary  reasons.  They  are  also  made  at  each  cf  the  DSARC 
decision  points:  at  DSARC  I to  "see  if  we  should  go  this 

way  and  if  it  is  cost  effective  s.t  DSARC  II  to  decide 

between  alternatives  and  for  resource  allocation;  and  at 
DSARC  III  as  a last  chance  to  say  no  on  the  project  or 
decide  hov/  many  to  buy.  In  the  production  phase  cost 
estimates  are  made  to  check  costs  of  numbers,  time/schedule 


and  technical  changes , or  to  estimate  what  changes  are 
necessary  if  the  ’oudget  is  adjusted.  .Another  purpose  for 
making  a cost  esti.mate  in  any  of  che  phases  of  the  acquisicicn 
cycle  is  for  nev/  concracts.  All  of  the  esximaxes  made  are 
perceived  by  the  ESD  3FC  esti.mators  as  being  used  chrcughouc 
the  chain  of  command  (S?0 , HQ  AFSC , HQ  A?  and  OSD)  for  the 
purposes  mentioned  earlier  in  this  paragraph  (50)- 

In  summary,  the  ESD  SPOs  matc.h  the  model  in  all  parts 
with  the  exception  of  the  techniq.ues  used  to  make  an  esti.mate. 
They  rely  heavily  on  the  estimates  made  by  the  contractor 
once  he  has  been  identified.  In  this  sense,  they  only 
adjust  an  already  completed  cost  estimate.  This  adjustm.ent 
is  based  on  a parametric  evaluation  of  the  contractor 


estimate  and  on  past  experience  with  the  contractor.  In  the 
conceptual  phase  they  match  the  model  by  using  the  RCA  PRICE 
model,  which  is  considered  a parametric  technio^ue.  In  the 
validation  and  full  scale  development  phases  they  do  net 
.match  the  model  because  they  use  para.metric  techniques  to 
check  the  contractors  estimate.  In  the  production  phase 
they  again  adjust  the  contractors  estimate  as  necessary. 

The  type  of  data  available  in  the  different  phases  does 
match  the  model  by  starting  as  vague  in  the  conceptual 
phase  and  building  to  historical  in  the  production  phase. 

The  purposes  of  the  estimates  in  the  different  phases  also 
matches  the  model.  Budgeting  and  official  SFC  estim.ates 


are  reouired 


;eptual,  validation  and  full  scale 
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development  phases,  while  cos':  analysis  estimaoes  are 
needed  in  the  production  phase  (50). 

Space  and  Missile  SvsTe.ms  Org'anication.  Cost 
estimating  for  new  major  weapon  systems  within  the  SAI'SC  SPCs 
follows  the  sequence  of  the  Weapon  System  Acquisition  Process 
Many  of  the  program  offices  contacted  reported  that  their 
programs  were  already  into  the  production  phase.  In  scm.e 
cases,  data  on  the  estimating  techir.iques  used  and  the  am.ount 
of  data  available  to  make  estimates  could  not  be  recon- 
structed. However,  cost  estimates  are  generated  following 
the  requirements  and  policies  of  the  weapon  system 
acquisition  process--in  the  conceptual,  validatio.n,  full 
scale  develcpm.ent  and  production  phases  (52). 

The  SAMuSO  SFGs  use  models  to  generate  their  cost 
estimates  in  all  phases.  Many  SFOs  do  not  have  cost 
estimating  persorcael  assigned  to  them  and  the  Directorate 
of  the  Comptroller  is  relatively  3mall--10  people  assigned 
(5I;  52).  In  most  SPCs  the  project  officers  and/or 
engineers  look  at  how  the  program,  is  defined,  what  the 
schedule  is  for  the  system.,  and  attem.pt  to  estimate  what 
equipment-- e . g. , electronics,  propulsion,  sensor,  etc.-- 
will  be  needed  to  meet  the  requirem.ents  of  the  weapon 
system.  They  also  attem.pt  to  obtain  data  from,  the 
contractor,  if  possible.  Using  this  data,  the  project 
officers  m.ake  an  estimate  of  what  the  system  will  cost. 
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This  technique  is  referred  to  at  SAI-iiC  as  an  Engineering 
Cost  Estimate.  It  is  developed  using  , at  different  times, 
techniques  from  all  four  categories  of  estimating  techniq,ues 
identified  in  the  model  (52). 

Concurrent  with  this  effort,  the  3?C  may  request  the 
Comptroller  (SAI.'ISO/.ACCE)  to  support  them  by  conducting  an 
Independent  Cost  Estim.ate  (ICE).'  The  ICE  is  generated 
using  a para.metric  cost  esti.mating  model.  S.AT.iC/ACCE  has 
developed  two  miodels  that  are  frequently  used--the  SAI.iSC 
Unmarj".ed  Spacecraft  Cost  Model  and  Schedule  Program 
Allocation  of  Resource  and  Cost  (3FARC).  The  SAI.5C  Tnmanned 
Spacecraft  Cost  Model  is  used  primarily  with  satillite 
systems  and  the  SPARC  model  is  used  for  launch  vehicles  cr 
boosters.  The  SFC  may  also  have  the  Aerospace  lorpcration 
(a  captive  contractor  who  provides  engi.neering  and  technical 
support  to  SAMiSO ) generate  a cost  estimate.  Aerospace 
Corporation  also  uses  a parametric  cost  estimating  model  to 
generate  the  cost  estimiate.  The  primary  difference  between 
these  m.odels  is  that  the  SAM.SG/ACCE  models  are  based  on 
subsystem  level  data  while  the  .Aerospace  Corpcraticn  model 
is  based  on  component  level  data.  In  either  case,  the  SFC 
must  provide  the  data  necessary  to  drive  the  m.cdel  used  (52). 

1 

“The  ICE  should  not  be  confused  with  an  IC.A,  v.'hich  is 
also  performed  by  SA.’1SC/ACCE , The  ICE  is  prepared  specifi- 
cally for  the  SPO  by  SA.''2C/ACCE  personnel  who  will  not  be 
involved  v/ith  the  IC.A.  The  ICA  is  used  to  check  the  reason- 
ableness of  the  SPO  estimate.  Independence  betv/een  the  SFC 
estim.ate  (.ACCE  generated  ICE)  and  the  ICA  is  preserved  (52). 
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The  SAT/ISC  Unmanned  Spacecraft  Ccsx  I'iodel  v/as 
developed  between  i960  and  I969.  and  was  revised  in  19?1. 
1972,  1973.  1975.  and  again  in  1977.  “he  model  u 


uses  caoa 


on 


seven  3ubsysterns--3trncoure , thermal  control,  cropulsion, 


lommunications , electrical  power  supply,  teler.erry 


racicirn 


and  command,  and  ohe  altitude  control  system.  The  m.odel  uses 

the  weight,  size  (volume)  and  number  of  units  reo.uired  for 

each  of  the  subsystems.  CERs  are  available  for  each  of  the 

subsystems  to  make  the  transition  from  historical  spacecraft 

2 

costs  to  conceptual  spacecraft  cost  estimates.  The  bFARC 
m.odel  was  developed  in  I968  for  use  with  launch  vehicles 
and  ICE.Ms . It  was  revised  in  1971  and  is  currently  'under 
revision  for  use  on  the  MC  IC3M  (51;  52). 

Cnee  the  5?Cs  have  the  engineer's  cost  estim.ate  and 
the  ICE  (from  either  or  both  SAI.SO/ACCE  cr  Aerospace 
Corporation)  , they  go  through  v/hat  is  called  ihe  normalizatior 
technicyues.  This  technique  consists  of  'updating  cr 
normalizing  the  ICE  by  the  'use  of  com.plexity  factors 
subjectively  derived  by  the  SFO  engineers  for  their 
particular  progra.m.  SAIISC/ACCE  also  provides  norm.alized 
CERs  for  'use  in  their  m.odel.  The  different  estim.ates  are 


2 

For  f'urther  i.nformaticn  on  the  S.A.1.S0  Un.manned 
Spacecraft  Cost  Model  and  the  cost  estim.ating  relationships 
and  norm.alization  factors , the  reader  is  directed  to  the 
following  SAMSO  documents:  5AMSG  Unmanned  Spacecraft  Cost 

Model,  third  Edition,  Uuly  1975  (SAI.'iSG  TR-75-229)  and  SA.VEG 
Un.T.aruced  Spacecraft  Cost  Model  Updated  Cost  Esti.mating 
Relationships  and  Ncrm.alization  Factors  (An  Interim.  Report) 
dated  Jan.  77. 
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then  examined  and  compared.  Any  differences  found  are 
resolved  between  the  project  officers  and  the  individuals 
performing  the  ICH.  In  this  manner,  the  official  program 
estimate  becomes  a composite  of  the  engineer's  cost  estimate 
and  the  ICE  (52). 

In  the  conceptual  and  validation  phases,  the  project 
off icer/engineer  relies  very  heavily  on  param.etric  techniques 
in  generating  the  cost  estimate.  In  the  full  scale  devel- 
opm.ent  phase,  contractor  data  becomes  available  for  the  first 
time,  facilitating  the  use  of  technic.ues  other  than 
parametrics . The  primary  reason  the  SAIvSC  SFCs  rely  upon 
the  parametric  cost  m.odels  in  the  conceptual  and  validation 
phases  is  because  the  weapon  systems  are  frequently  one-of- 
a-kind  programs  that  are  pushing  the  state-of-the-art  in 
technology.  Little  useable  data  is  available  from  previous 
programs.  The  data  used  to  drive  the  models  ranges  from 
estimated  engineering  data  (using  analogies  to  other  program.s 
v/henever  possible)  in  the  conceptual  and  validation  phases 
to  actual  production  data  in  the  production  phase.  Therefore, 
SAT'ISC/ACCS  developed  the  models  presented  above  to  help  use 
the  data  points  from  previous  programs.  These  models  are 
under  continual  review  and  are  updated  freo^uently  (52). 


The  SAI'SC  S?Cs  generate  cost  estimates  for  a variety 


contingency  planning  purposes  and  in  response  to  "v/hat  if" 
questions  from  higher  headquarters.  S?0  cost  estimates  are 
used  in  planning,  oontrolling,  budgeting  and  decision  making 
at  all  echelons  of  command  authority  from  the  SrC  director  to 
the  Secretary  of  Defense,  Congress  and  the  President  (5S). 

In  summary,  the  SAI.5C  SPOs  do  validate  the  model,  as 
developed  in  Chapter  II,  in  all  areas  except  for  technique 
used.  Official  SPG  estimates  are  made  in  each  phase  of  the 
acquisition  cycle--conceptual , validation,  full  scale 
development  and  production.  The  data  available  for  making 
cost  estimates  corresponds  to  the  amount  of  data  available 
that  was  predicted  for  eaoh  phase.  Data  availability  grow' 
from  little  or  no  useful  data  (vague)  to  hard,  actual 
production  data  (historical)  as  the  weapon  system  is 
conceived,  defined,  tested  and  produced  for  operational  use. 
The  technique  used  to  generate  the  cost  estimate  progresses 
from  parametric  techniques  in  the  conceptual  and  validatic" 


;ha.^es  to 

develoom 

;echniques 

in  the  full 

scale  de' 

tandards 

techr.ique  in 

the  prod' 

ia.'or  Comm 

.and  ICAs 

In 

dec  end  er.t  Cos 

t Estima 

• irection 

of  the  Com.ptroller , >1 

;;  .AFLC  Co 

m,ptroller  is 

respons i 

ICAs  on  major  AF  weaqon  system  trcarams  that  are  under  the 


d 


con'rol  of  AFLC . These  are  prograr.s  thax  involve  major 
modif icaXicns  to  existing  weapon  systems.  The  HQ  AFSC 
Comptroller  is  responsible  for  generating  ICAs  on  major 
weapon  system  programs  that  are  under  the  control  of  AFSC, 
i.e.,  new  weapon  system  acquisitions.  The  ICEs  generated  by 
these  organizations  are  used  within  the  Independent  Cost 
Analysis  program,  v/hich  supports  the  .AFS.A.RC  and  LShRC 
programs  (33;  43). 

HQ  .IFLC  IC.A.  The  Com.ptroller , HQ  AFLC,  m.axes  at 
least  one  cost  estimate  in  each  of  the  first  three  phases 
of  the  acq^uisition  cycle.  The  ICAs  are  acccmiplished  just 
prior  tc  the  LSA.RC  at  the  end  of  the  conceptual,  validaticn 
and  full  scale  development  phases  (33). 

In  the  conceptual  phase,  the  estim.ators  use  general 
performance  parameters  specified  for  the  modif ication.  The 
estimators  also  use  data  from  programs  with  similar 
performance  parameters.  In  the  validation  phase  the  same 
type  of  data  is  used.  However,  the  data  is  tetter  defined 
and  in  more  detail.  Design  specifications  and  engineering 
"build-up"  data  (specifications  and/or  costs  for  subass em.- 
blies)  are  used  in  the  full  scale  development  phase  (33). 

The  .\FLC  Comptroller  cost  estimators  indicated  that 
" . . . techniques  used  are  limited  only  by  the  available 
data  [33’’."  The  techniques  that  are  used  in  the  conceptual 
phase  are  analogies  and  CERs . In  the  validaticn  prase 
analogies  and  CERs  are  again  used,  with  the  addition  of 
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build-up/grass  roots  and  s-catistical  techniques  when 
possible.  The  estimators  use  models  such  as  the  iiCD  I-ietric 
Model  and  the  Logistics  Support  Cost  Model  in  the  full  scale 
development  phase.  These  models  are  used,  in  conjunction 
with  the  parametric  and  grass  roots  techniques , if  the 
proper  data  is  available.  The  individual  estim.ator  choses 
the  technique  which  he  perceives  as  best  suited  for  use 
with  the  available  data  (38). 

The  Comptroller,  HQ  AFLC , makes  cost  estimates  in 
support  of  the  ICA  program.  The  ICA  is  generated  to  check 
the  validity  and  reasonableness  of  the  3TC  estimate.  The 
ZCA  is  reviewed  by  the  AF  CAIG  and  CSD  CAIG  in  support  for 
the  AFSARC  and  LSAF.C  programs  (38). 

HQ  AFS C ICA.  AFSC  Independent  Cost  Estimates  are 
generated  by  the  Directorate  of  the  Ccm.ptroller , Aeronautical 
System.s  Division  (ASD)  , Electronics  Systems  Division  (ESD)  , 
and  Space  and  Missile  Systems  Organization  (SAJ-iSC).  These 
Independent  Cost  Estimates  are  used  within  the  .IFSC  Inde- 
pendent Cost  Analysis  (ICA)  program.  Each  organization 
has  developed  and  implemented  an  ICA  program  to  support 
AFSC  IC.^  requirements  (43:  46;  49;  5i ) • These  crgar.izaticn 
ICA  programs  are: 

1.  Aeronautical  Systems  Divis ion--ASD , as  part  cf 
its  ICA  program,  prepares  an  Independe.nt  Cost  Estimate  (ICE) 
for  each  DSARC  review  of  a program  controlled  by  an  AST  SFC . 
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An  ICA  is  prepared  ad  the  er.d  of  the  conceptual,  validation 
and  full  scale  development  phases.  The  testing  required  in 
the  full  scale  development  phase  may  not  he  ccmipleted  v/hen 
the  DSASC  meets.  To  prevent  a time  gap  hetween  the 
development  and  production  of  the  prototype  weapon  system 
and  the  beginning  of  production  of  the  operational  weapon 
system,  the  DSA3C  meets  to  decide  if  a limited  production 
should  begin  while  testing  continues.  The  lead  time  between 
the  end  of  prototype  production  and  operational  production 
can  run  as  long  as  ij  years.  Delaying  production  until  all 
testing  is  completed  could  severely  hamper  production 
schedules  and  raise  production  costs.  Therefore,  two 
estimates  are  frequently  made  for  the  full  scale  development 
phase  (DSARC  III  A a.nd  B5„RC  ill  3)  (D6). 

The  techniques  used  to  generate  the  cost  estim.ate 
are  closely  related  to  the  amount  of  data  available.  The 
.ASD  ICA  program  uses  parametric  estimating  techniques  for 
the  conceptual  and  validation  phase  estim.ates.  Cost  m.cdels  , 
analogies,  CERs  and  regression  analysis  are  exa.mples  of  the 
param.etric  techniques  used  in  these  phases.  In  the  full 
scale  development  phase,  the  .ASD  ICA  program  uses  a 
"bottoms  up"  technique.  This  technique  is  a combination 
of  the  parametric  techniques  used  earlier  and  a grass  roots 
aoproach  using  the  work  breakdown  structure  of  the  Cost/ 
Schedule  Control  Systems  Criteris  (C/SCSC).  This  technique 
is  also  used  in  the  validation  phase  whenever  pcssible  (^6). 
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The  purpose  of  the  estimate  generated  'ey  rhe 
Directorate  of  the  Comptroller,  ADD,  is  to  provide  a check 
of  the  methodology  and  reasona'oleness  of  the  System  Program 
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CAIG,  and  G3D  GAIG  with  an  independent  view 


the  proposed  weapon  system.  The  ICA  estimate  is  reviewed 
at  the  APSARC  and  t.he  DSARC  (46). 

2.  Electronics  Systems  Divis ion--E3D  has  an 
Independent  Cost  Analysis  program  similiar  to  the  ASD 
program.  Cost  estimates  are  mads  at  the  same  three  points 
in  the  Weapon  System  Acquisition  ?rocess--conceptual , 
validation,  and  full  scale  development  phases.  E2D  also 
makes  two  ICAs  during  the  full  scale  development  phase  when 
the  weapon  system  involved  has  a long  lead  time  'cetween 
prototype  development,  production  and  testing  and  operational 
system  production  and  deployiment.  The  second  estimate 
(for  DSA.HC  III  3)  is  considered  'oy  EED/ACCE  personnel  to  he 
an  update  of  the  estimate  made  for  DSARC  III  A rather  than 
a completely  new  estimate  (h-9), 

ESD  makes  extensive  use  of  parametric  cost  estimating 
techniques  in  all  ICA  estimates.  Parametric  techniques 
(analogies  to  other  systems  primarily)  are  used  to  estimate 
the  cost  of  'nardware  and  software  configurations  that  will 
'oe  reojuired  in  the  weapon  system.  The  List  Price  cr 
Catalogue  Price  technique  is  then  used  to  price  out  the 
electronic  components.  The  electronics  used  in  radar  and 
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conmunications  weapon  systems  are  basically  off-the-shelf 
icems,  which  lend  themselves  to  the  use  of  che  list  price 
technique.  The  RCA  PRICE  model  is  used,  in  conjunction  with 
the  list  price  technique,  to  generate  the  cost  estimate.  In 
the  full  scale  development  phase,  the  grass  roots  or  detailed 
engineering  technique  is  frequently  used.  A parametric 
technique  is  also  used  to  generate  a backup  estimate  and  to 
give  credence  to  the  grass  roots/detailed  estima.e.  E£D 
estimators  frequently  attempt  to  estimate  the  costs  of  a 
weapon  system  using  two  different  methods  for  comparative 
purposes  (b9)- 

The  ESD  ICA  is  used  by  the  Air  Staff,  AF  CAIC 
(AFSARC),  and  CSD  CAIC  (DSARC)  as  a confirmation  check  on 
the  methodology  and  accuracy  of  the  official  5?C  estimate. 

It  is  also  used  to  verify  the  SPC  estimates'  validity  and 
reasonableness  (Sp). 

3.  Space  and  Missile  System.s  C rganizaticn--The 


at  ASD  and  ESD.  SAT-ISC/ACCE  also  makes  cost  estim.ates  for 
each  of  the  DSARC  reviews  at  the  end  of  the  conceptual, 
validation  and  full  scale  development  phases  (5l)- 

3AMS0  estimators  are  hampered  by  the  lack  of  an 
extensive  data  base,  more  so  than  their  ASD  a.nd  ESD 
counterparts , A common  complaint  was,  "...  data  on 
parameters  other  than  weight  and  power  weren't  collected 
A second  reason  is  that  not  as  many  space  systems 
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have  been  developed  and  produced  as  have  aeronautical  or  | 

1 

radar/communications  systems.  In  addition,  space  systems  i 

are  not  produced  and  deployed  in  large  numbers.  Each  srace 

1 

system  is  substantially  different  from  any  other  space 

system.  SA.V.3C  cost  estimators  are  able  to  use  standard, 

actual  costs  for  launch  vehicles.  The  Titan  launch  vehicle 

is  used  to  launch  most  spacecraft  systems  and  the  costs  are  ^ 

well  known  from  historical  data.  This  reduces  the 

I 

requirement  for  new  estimates  of  this  rather  costly  pcrticn 
of  the  overall  weapon  system  (51). 

Parametric  estimating  techniques  are  used  in  all  three 
phases  by  the  SAIvEC  estimators.  Extensive  use  is  made  of 
cost  models  which  have  been  developed  by  SAIvIScyACCE . The 
particular  model  used  is  dependent  upon  the  type  of  program, 
or  portion  of  the  program  being  estimated  (51).  The  5A.V.SC 
Unnamed  Spacecraft  Cost  Model  is  used  on  all  spacecraft 
and  communications  progra.ms.  Estimating  the  cost  of  sensors 
for  the  different  spacecraft  is  very  difficult  because  of  a ! 

lack  of  experience  in  the  area  and  the  wide  range  cf 
sensors  used.  Analogies  use  of  data  from,  other  program.s  , 

3 

and  the  use  of  the  .A.CCE  models  represent  attempts  to 

[ 

overcome  these  problems.  The  RCA  PRICE  model  and  catalogue  i 

data  are  used  to  estimate  the  costs  of  the  ground  station  ; 

portion  of  the  weapon  systems  and,  as  stated  previously, 
actual  production  costs  are  used  to  estim.ate  the  launch 
vehicle  portion  cf  the  v/eapon  system..  As  better  and  m.ore 
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factual  data  becomes  available  in  the  full  scale  development 
phase  of  the  program,  it  is  used  to  drive  the  m,odel.  The 
mere  factual  data  replaces  the  earlier  estimated  data  which 
was  used  in  the  model  (5I). 
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the  Air  Staff,  A?  CAIG,  and  C5D  CAIG  with  an  independent 
view  of  what  the  program  costs  for  the  weapon  system,  is 
estimated  to  be.  The  IGA  estimate  is  used  in  the  ArSARC 
and  DSARC  reviews  (51). 

Summary.  The  AFLG  and  A.FSG  (A.SD,  EST,  and  SAiiSC ) 
IGA  programs  all  m^atch  the  miCdel  in  the  areas  of  when  cost 
estimates  are  made,  the  amount  of  information  available  for 
making  estim.ates,  and  the  type/purpose  of  cost  estim.ates 
made.  The  SAI-iSC  urogram  conform.s  to  the  m.cdel  in  the  area 
of  technio.ues  used.  However,  the  IGA  programs  at  AFLG, 
ASD  and  BSD,  do  not.  These  organizations  use  grass  roots/ 
detailed  engineering  (Research  and  Developm.ent  Techniques) 
techniques  in  the  validation  and  full  scale  development 
phases  whenever  possible,  as  well  as  parametric  techniques, 
to  generate  their  IGA  estim.ates  (33;  1-3;  ^6;  51).  The 

rationale  behind  this  difference  is  that  to  get  the  best 
(micst  accurate)  estimates,  the  available  data  (whether 
parametric  GER's  or  engineering  actuals)  should  be  used. 
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Justification  for  this  rationale  comes  from  AF  Reg  173-ii 
paragraph  4. a (35) « 

AF  CAIG 

The  AF  CAIG  was  not  included  in  the  model  of  the  Air 
Fores  Cost  Estimating  Process  v/ithin  the  Air  Force  Weapon 
System  Acc^uisition  Process  developed  in  the  literature  review. 
It  was  identified  as  a potential  source  of  cost  estimates 
during  the  data  collection  phase  of  this  research  effort, 
and  was  therefore  investigated  as  part  of  the  research 
effort.  HQ  Operating  Instruction  173-3.  dated  26  June, 

197^  "...  establishes  the  AIR  Force  CAIG  v/ith  membership 
and  responsibilities  parallel  to  those  of  the  CSD  CAIG 
i_5S:l'!."  The  primary  responsibility  of  the  AF  CAIG  is  to 
" . . . act  as  an  advisory  tody  to  the  Secretary  of  the  Air 
Force  for  Financial  Rianagement  and  the  Comptroller  of  the 
Air  Force  on  all  matters  related  to  weapon  systems  costs 
to  be  presented  to  the  CSD  CAIG  l53:1j."  The  .IF  CAIG 
provides  the  .Assistant  Secretary  of  the  Air  Force  for 
Financial  Management  (SAF/FM)  with  cost  issue  summaries  and 
an  evaluation  and  review  of  ICAs  prior  to  their  submission 
to  the  CSD  CAIG.  The  SAF/FM  uses  the  AF  CAIG  report  at  the 
meeting  of  the  AF  System  Acquisition  Review  Council  (.AFS.ARC), 
a review  body  similiar  in  composition,  responsibilities  and 
operation  to  the  DSARC  (see  page  12)  (30:2). 

Additional  AF  CAIG  responsibilities  include: 

1.  Assessing  and  making  recommendations  concerning 
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all  cost  objectiyes  contained  in  Decision  Coordinating 
Papers  (DCPs)  and  Program  Decision  memoranda  (FDI»'s)  before 
they  are  submitted  to  OSD. 

2.  Accomplishing  cost  studies  and  other  cost  related  3 

tasks  as  requested  by  Secretary  of  the  Air  Force  or  Chief  of 
Staff  of  the  Air  Force. 

3.  Conducting  direcx  efforts  v/hich  will  improve  xhe 
cost  esximaxing  capability  of  the  Air  Force  and  reporting 
periodically  to  the  SAF/FM  and  Comptroller  concerning  these 
efforts  ( 5S : 11 ) . 

KQ  USAF/ACM  is  tasked  to  prepare  a proposed  AF  CAIC 
report  which  includes  a summary  of  the  cost  issues  that  will 
be  addressed  by  the  AF  CAIG.  This  report  and  the  cormtand 
ICA  are  reviewed  and  evaluated  prior  to  the  submission  of 
the  ICA  to  the  CSD  CAIC-.  The  proposed  report  functions  as 
the  basis  of  the  final  AF  CAIG  report,  which  incorporates 
the  comments  and  recomjnendations  of  the  AF  CAIG,  summarises 
all  cost  issues,  and  provide  the  AF  CAIG '3  overall  evaluation 
of  the  command  ICA  (58:1). 

In  depth  interviews  with  members  of  the  .^F  CAIG 
revealed  that  the  AF  CAIG  accomplishes  a fcrmial  review  and 
evaluation  of  all  command  ICAs  prior  to  the  submiission  of 
the  ICA  to  the  CSD  CAIG.  In  conjunction  witn  this  review 
and  evaluation,  the  AF  Comptroller  (ACIvi)  may  m,ake  a cost 
estim.ate  to  check  the  validity  and  reasonableness  of  the 
comm.and  ICA  and  SPG  estim.ates.  The  review  and  evaluation 
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of  Command  ICAs  take  place  in  the  conceptual,  validation 
and  full  scale  development  phases  of  ixs  weapon  sysxem 
acq^uisition  cycle.  The  AF  CAIC-  also  makes  cost  estimates 
and/or  reviews  and  evaluations  for  DCFs  and  rDI.is . They  also 
make  cost  estimates  in  response  to  congressional  inquires. 
These  activities  can  and  do  occur  at  any  time  during  the 
weapon  system  acquisition  cycle  (37). 

A?  CAIG  estimators  pointed  out  that  the  amount  and 
type  of  data  available  for  their  use  limited  the  choice  of 
techniques  they  can  use.  They  employ  models,  CERs , lab 
studies,  regression  analysis,  parametrics,  and  bottoms  up 
or  grass  roots  techniques.  In  most  cases,  however,  the 
estimators  are  limited  to  the  use  of  parametric  techniques . 
They  attempt  to  replace  the  analogous  data  with  actual  data 
as  it  becomes  available  (37). 

The  individual  estimators  also  felt  that  they  were 
restricted  to  the  use  of  parametric  techniques  because 
of  time  limitations.  The  review  and  evaluation  of  the 
command  IGA  and  the  generation  of  a new  cost  estimate,  when 
required,  must  normally  be  accomplished  within  five  (5) 
working  days  between  the  receipt  of  the  ICA  and  the 
preparation  of  the  proposed  report.  This  tim.e  factor 
prevents  the  estimators  from  using  more  detailed  techniques, 
such  as  bottoms  up  or  grass  roots,  which  require  mere  time  to 
complete  than  is  available  (37). 

Another  limiting  factor  is  the  number  of  personnel 
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assigned  to  perform  the  review  ar.d  analysis  function. 

Presently  only  six  people  are  assigned  as  cost  analysts 
and  Program  Cost  Monitors  (PCKs).  These  people  must  perform, 
t.he  cost  analyses  on  the  ICAs  and  prepare  new  esoim.ates  if 
the  facts  warrant.  They  also  perform  the  review  and 
evaluation  for  other  cost  esti.mates  outside  the  DSAP.C  process 
that  the  A?  CATC  requires  (37). 

The  primary  reason  for  the  AF  CAIG  is  to  support  the 
SAF/FM  with  inform.ation  for  making  program  decisions.  The 
AF  CAIG  looks  for  agreements/disagreements  between  the  SPC 
estimate  and  t.he  command  ICA.  If  there  is  a major 
disagreement  between  these  estimates,  the  AF  CAIG  ccmm.ents 
on  the  reasons  for  the  differences  and  makes  recommendations 
to  the  S.AF/FII.  The  SAF/FM  .ma.kes  the  decision  on  whic.h 
estim.ate,  the  comjnand  ICA  or  the  -A?  C.IIG  estimate,  to 
forward  to  the  CSD  CAIG.  Only  one  estimate  will  be  forwarded. 
If  there  are  no  major  disagreements , the  comjr.and  ICA  estim.ate 
will  be  forwarded  to  the  SAF/FM  (3?). 

The  AF  C.IIG  also  do  their  own  review  and  evaluation, 
and  make  cost  assessments  or  recomuriendations  on  DCPs  , 

PDKs , or  in  answer  to  congressional  inquires.  These  cost 
estimates  and/or  assessments  and  recommendations  are  used 
for  planning  in  all  phases  of  the  weapon  system  acquisition 
cycle  by  the  SAF/FM,  members  of  the  .Air  Staff,  a.nd  the 
Progra.m  Eleme.nt  .Monitor  (37)- 

In  summary,  the  AF  CAIG  m.akes  cost  estim.ates  . and/or 
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reviews  and  evaluations  of  command  ICAs  in  the  first  three 
phases  of  the  weapon  system  acquisition  cycle  (conceptual, 
validation  and  full  scale  development).  They  also  make  cost 
estimates,  assessments  and  recommendations  based  on  estimates 
generated  by  other  organizations  in  these  three  phases  and 
in  the  production  phase.  The  data  used  to  make  these 
reviews  and  evaluations  (and  new  cost  estimates  when  neces- 
sary) increases  in  quality  and  quantity  as  the  weapon  system 
progresses  through  the  acquisition  cycle.  The  AF  GAIT  is 
limited  to  the  use  of  parametric  cost  estimating  techniques 
because  of  the  amount  and  type  of  data  available,  the  amo'unt 
of  time  available  to  conduct  the  review  and  limited  number 
of  persormel  assigned.  The  reports  generated  by  the  AF 
CAIG,  their  reviews  and  evaluation  of  the  ICAs  and  new  cost 
estimate,  if  req^uired,  are  used  by  the  SAF/FI<I  to  make 
program  decisions  prior  to  submission  of  the  IC.'^  and  SFO 
estim.ate  to  the  CSD  CAIG.  The  findings  of  the  AF  CAIG 
are  used  by  the  CSD  CAIG  in  making  their  recommendaticns 
to  the  DSAFC . The  assessments  and  recommendations  made  by 
the  AF  CAIG  separate  from  the  DSARC  process  are  used  for 
planning  and  budgeting  by  the  3AF/FM,  members  of  the  Air 
Staff,  and  rrogram  Element  Monitors.  They  are  also  used 
to  respond  to  congressional  inquires  (37). 

CSD  CAIG 

The  CSD  CAIG  reviews  the  ICA  and  SrC  estimates 

prior  to  each  DS.A.RC  review.  It  provides  the  DSARC  with  an 

76 


a recommendation  on  all  cost  objec 


assessment  of  and/or 
In  most  instances,  the  C5D  CAIG  does  not 


of  its  own.  If  in  their  opinion  a major  item  has  been 
misinterpreted  or  left  out  of  a cost  estimate,  however,  they 
will  make  their  own  estimate  and  forward  it  to  the  DSARC 


along  with  the  ICA  and  3?C  estimates.  The  CSD  CAIG,  in 
fulfilling  its  responsibility  to  "accomplish  other  tasks 
and  specific  studies  as  requested  by  the  DIARC  principles 
j_32";"  also  provides  cost  estimates  tc  answer  questions  from 
the  DSARC  on  such  issues  as  cost  growth  (32) • 

The  techniques  available  to  the  CSD  CAIG  are  lim.ited 
by  time  and  manpower  availability.  The  CSD  CAIG  normally 
has  only  nine  days  in  which  to  review  the  documentation 
provided  to  them.  In  this  time  fram.e  they  must  analyze  the 
data,  make  a cost  estim.ate  (if  required)  and  provide  their 
report  to  the  DSARC.  Two  individuals  assigned  tc  the  CSD 
CAIG  are  primarily  responsible  for  the  cost  analyses  of 
all  major  AF  weapon  system  acq.uis itions . Therefore,  the 
technic^ues  used  are  parametric  in  all  phases  (32). 

The  cost  esti.mates  and/or  reviews  of  the  ICA  and 


SrO  cost 

estimates  made 

by  the 

CSD 

CAIG  are 

provided  to  the 

DSARC  in 

the  form  of  a r 

■eport . 

Thi 

■ S report 

is  used  b;/  the 

DSARC  in 

making  recommen 

.dations 

on 

resource 

allocation : 

that  is , 

whether  or  net 

the  weapon 

system  sh 

culd  proceed  tc 

are  used  for  plarjaing  ard  budgeting  by  both  the  ECf  and  the 
Air  Force.  The  OSB  CAIG  report  is  also  sent  to  the  AF  CAIG, 
where  the  cost  explanation  and  analysis  can  be  used  to 
improve  the  future  cost  estimates  (32). 

In  summary,  the  CSD  CAIG  validates  the  model.  They 
evaluate  and  review  the  3?C  and  independent  cost  estim.ates. 
They  also  make  an  actual  cost  esti.mate  when  requested  to  do 
so  or  when  they  believe  one  is  required.  The  CSD  CAIG 
provides  the  review  and/or  estim.ate  just  prior  to  the  DSARC 
reviews.  The  most  current  data  available  is  used  in  the 
analysis  of  the  SPC  and  ICA  estim.ates.  This  data  was 
perceived  to  grow  in  quantity  and  quality  as  the  weapon 
system  progresses  through  the  acquisition  cycle.  The  data 
grows  from  very  little  in  the  conceptual  phase  to  som.e  cost 
actuals  in  the  full  scale  development  phase  (32). 

It  should  be  noted  that  persorctel  from  the  CSD  CAIG 
have  found  one  "discrepency"  in  our  description  cf  them  on 
page  15  of  this  thesis.  "The  CSB  CAIG  does  not  resolve 
differences  between  the  ICA  and  SFC  estimates.  They  only 
extilain  the  differences  [_32  . " With  this  in  mind,  they 
believe  that  they  have  no  advocacy  role  to  play  and  thereby 
perceive  their  estimates  and  evaluations  as  more  objective 
than  anyone  else's.  They  also  believe  that  this  cb j ectiveness 
is  the  reason  their  estimate  changes  less  than  the  SPC  and 
ICA  estimates  during  the  acquisition  cycle  (32).  Their  data 
indicates  that  the  SPC  and  ICA  estimates  tend  to  approach 
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the  less  optimistic  CSD  CAIl  estimaxes  as  a v/eapon  system 
progresses  through  ixs  weapon  system  acquisition  cycle. 


Research  Questions 
Answered 


Research  Question  1.  V/hat  model  of  th.e  cost 
estimating  process  for  new  major  Air  Force  weapon 
systems  can  be  developed  that  depicts; 

a.  the  organizaticn/agency  providing  each  cost 

0S  z 1 6 

b.  the  purpose  for  v/hich  each  cost  estimate  is 
developed , 

c.  the  point  in  the  acquisition  process  where  these 
estimates  are  made,  and 

d.  the  tecliniques  which  are  appropriate  to  each 
organization/agency  in  producing  their  estimate, 
considering  the  information  available  at  the  point 
'When  the  estimate  is  reeded? 

A model  to  answer  research  question  1 was  originally 
developed  in  Chapter  II  based  on  m.anagement  theory  and  a 


revie'w  oz 


it  13  pres  enter,  in 


the  available  literature, 
figure  2 (pages  37-33).  Curing  the  course  of  the  field 
interviews  to  validate  the  model  and  answer  research 


auestion  2, 


the  AF  CAIG  'was  identified  as  a source  cf  cost 


estimates.  An  "Updated  Model  of  the  Air  Force 
Estimating  Process  -Vithin  the  Air  Force  Weapon 
Acquisition  Process"  is  presented  in  figure  3. 
model  includes  the  AF  CAIG  and  identifies 
to  the  AF  cost  estimating  Torocess. 


Cost 


.nis  upcateo 


Its  con-ripution 


Research  Question  2.  If  such  a m.cdel  can  be 
developed,  can  it  be  validated  to  demonstrate  that 
it  realistically  reflects  the  "real  'world"  estim.ating 


on  V ii'  0 rjTi  s r.  “ 


:he  agencies  concerned  wit: 


ac'juisiOion  of  nev/  major  mr  rcrce  \veapcn  sysoemsr 
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Fig-ure  3 (oontinued) 


Extensive  field  interviews  with  cost  estimators 


from  each  organization/agency  identified  in  the  literature 
review  were  conducted  to  answer  research  question  2.  The 
results  of  these  interviews  were  analyzed  above  and  a new 
model  was  developed  to  reflect  what  actually  happens  in  the 
process  of  estimating  the  cost  of  a major  weapon  system 
(see  figure  ^) . The  major  differences  found  between  the 
model  based  on  the  literature  review  (figure  3)  and  the 
real  world  model  (figure  4)  are  described  below: 

1.  The  techniques  used  by  the  System  Program  Offices 
do  not  progress  from  Parametric  (conceptual  phase)  to 
Research  and  Development  (validation  phase),  Engineering 
(full  scale  development  phase),  and  Standards  (production 
phase)  as  predicted  by  the  literature  model.  Instead,  the 
SPOs  use  Parametric  techniques  in  both  the  conceptual  and 
validation  phases.  The  data  increases  from  vague  to  limited 
during  progress  through  these  phases,  but  is  still 
apparently  not  sufficiently  detailed  cr  reliable  to  support 
the  use  of  techniques  more  sophisticated  than  those  in  the 
Parametric  category. 

2.  The  techniques  used  to  develop  the  ICA  in  the 
validation  and  full  scale  development  phases  are  also 
different  in  the  two  models.  The  ICA  estimators  use 
Parametric  techniques  in  all  three  phases  (conceptual, 
validation  and  full  soale  development).  When  possible,  they 
also  attempt  to  use  Research  and  Development  techniques  in 
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Real  V/orld  Model  of  the  Air  Force 
Cost  Estimating  Process  Within 
the  Air  Force  Weapon  System 
Acquisition  Process 
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The  contractor  & SPO  can  make  mors  than  one  estimate  during 
each  phase.  An  estimate  is  made  and  revised  until  time  of 
presentation  to  the  DSARC  for  a decision. 

** 

The  SPO  & Contractor  personnel  make  continual  on-going 
cost  estimates  throughout  the  production  phase. 


Techniques  actually  used  will  vary  from  program  to 
program  and  depend  upon  the  type/amount  of  data  actually 
available  at  the  time  the  estimate  is  made. 

Following  DSARC  III  the  AF  CAIG  and  CSD  CAIG  monitor  the 
cost  estimate  and  respond  to  inquiries  concerning  program 
costs/estimates . 


Figure  4 (continued) 
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the  validation  and  full  scale  development  phases.  When  the 
Research  and  Development  techniques  are  used,  they  are 
combined  with  Parametric  techniques.  The  data  available 
increases  in  both  quantity  and  detail  from  one  phase  to  the 
next  in  all  programs.  However,  it  is  not  always  sufficiently 
developed  or  available  to  allow  the  use  of  Research  and 
Development  estimating  techniques.  Aeronautical  and 
electronic  weapon  systems  are  developed  frequently  enough 
that  a better  historical  data  base  exists  from  which  to 
predict  costs  (41).  This  accounts  for  the  use  of  Research 
Develcpment  techniques  in  ICAs  generated  for  these  programs. 
In  all  cases.  Parametric  techniques  are  used  in  combination 
with  the  Research  and  Development  techniques. 

General  Imoressions 

While  conducting  interviews  of  cost  estimators 
representing  all  estimating  agencies  from  the  SPOs  up  to 
and  including  the  OSD  CAIG,  it  was  impossible  not  to  derive 
some  general  impressions  of  the  cost  estimating  process. 

These  observations  are  summarized  below: 

1.  DCD  organizations/agencies  identify  the  need 
for  new  weapon  systems,  define  the  type  of  system  required  to 
fill  the  need,  and  defind  the  system  and  its  f^-inding  before 
Congress.  It  would  be  unrealistic  to  expect  DCD  agencies 
tc  develop  weapon  system  cost  estimates  in  any  but  the  most 
favorable  light  possible.  The  SPO  acts  as  the  official 
progra.m  advocate.  As  such,  their  estimates  tend  to  be 
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"1 

optimistic  is  an  attempt  to  sell  the  program  to  AFLC/AFSC, 
the  Air  Staff,  DOD,  and  eventually  to  Congress  (4?;  5O;  52). 

The  ICA  estimates  are  used  to  check  the  reasonableness  of 
the  3F0  estimate.  The  ICA  estimate  is  generally  relatively 
close  to  the  3P0  estimate.  This  is  to  be  expected  since 
both  estimates  are  generated  from  the  same  data  base  (38:  43: 

46;  49:  5i ) . The  AF  and  OSD  CAIGs  normally  only  compare  the 
SPO  estimate  with  the  ICA  estimate  to  insure  that  no  major 
discrepancies  or  differences  occur  that  cannot  be  explained. 

The  CAIGs  also  check  to  insure  that  the  methodologies 
employed  by  the  SPO  and  ICA  estimators  are  adequate  and 

appropriate  (32:  37).  In  fact,  our  information  indicates  ; 

that  the  SPO,  AFLC/AFSC  ICA,  and  the  AF  and  CSD  CAIGs  all  | 

act  in  varying  degrees  as  program  advocates  in  the  majority  j 

of  weapon  systems  acquisition  cases.  The  “endency  is  to  j 

present  the  cost  estimates  for  weapon  systems  in  a 
favorable/optimistic  light  which  allows  the  program  to 
continue  through  eventual  acquisition  to  an  operational 
sratus.  This  viewpoint  is  supported  in  the  literature  also. 

J.  Ronald  Fox,  a former  Assistant  Secretary  of  the  Army,  in 
his  book.  Arming  America;  How  the  U.  5.  Buys  Weapons, 
expresses  the  thought  that  all  levels  of  management  within 
the  XD  attempt  to  provide  overly  optimistic  estimates  to 
decision  makers  (10:159-164). 

2.  The  organizations  apparently  have  few  or  no  ; 


mechanisms  established  which  would  motivate  cost  estimators 
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to  develop  estimates  that  accurately  reflect  the  eventual 
total  cost  of  the  weapon  system.  A large  number  of  estimators 
perceived  that  pressure  was  exerted  for  them  to  generate  an 
estimate  that  was  "reasonable  and  sellable".  Some 
individual  estimators  felt  that  their  superiors,  as  well  as 
personnel  at  higher  headquarters,  were  primarily  interested 
in  obtaining  an  optimistic  (i.e.,  a low)  estimate,  one 
which  was  close  to  previous  estimates  made  on  the  program 
and  one  which  frequently  reflected  a need  to  achieve  an 
optimal  pattern  of  "luck"  on  all  program  activities  (46; 

4?:  49;  50;  51:  52).  Fox  also  supported  this  view.  He 
commented  that  program  managers  frequently  perceived  that 
they  had  great  difficulty  transmitting  reasonable  estimates 
to  higher  headquarters.  They  reported  that  higher  level 
managers  tended  to  reduce  the  cost  estimates  to  a low,  more 
optimistic  figiare  (10: 160).  This  is  not  to  say  that  any 
estimator  or  program  manager  falsifies  any  cost  estimate. 

The  estimator  is  merely  internally  motivated  by  the 
organization  to  provide  an  estimate  which  will  help  sell 
the  program,  one  which  assumes  everything  questionable  will 
turn  out  well  for  the  program. 

3.  Each  organization  involved  in  the  study  iden- 
tified the  lack  of  an  adequate  data  base  as  a major  problem 
faced  by  the  individual  estimator.  The  data  in  those  data 
bases  that  do  exist  is  not  standardized  from  one  program 
to  another,  nor  is  the  methodology  used  to  generate  past 
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estimates  included  in  the  data  base.  As  one  estimator 
put  it,  "This  makes  the  entire  data  base  worthless,  to  say 
the  least."  This  feeling  that  the  data  bases  which  are 
available  are  of  little  value  prevailed  in  many  of  the 
organizations . 

4.  The  authors  encountered  a major  problem  with 
the  semantics  involved  in  the  techniques  used  by  the 
estimators.  Many  estimators  had  different  conceptions  as 
to  what  the  techniques  actually  were,  or  used  different 
terminology  for  the  same  technique.  The  semantics  problem 
occured  in  almost  every  organization  involved  in  this 
research  effort  and  was  evident  between  the  individual 
estimators  v/ithin  an  organization.  This  confusion  was  also 
noted  in  the  literature  (reference  pages  22  to  36). 

5.  The  individual  estimators  are  not  held  accountable 
for  the  accuracy  of  their  estimates.  In  no  organization 
visited  could  the  authors  establish  the  existence  of  a 

track  record  of  an  individual's  estimates.  In  fact,  the 
majority  of  the  cost  estimators  were  provided  no  feedback 
relative  to  their  estimates  on  either  an  individual  or  a 
program  basis.  No  feedback  on  the  individual’s  estimates 
were  formally  made  available  to  him.  No  data  on  estimates 
made  by  other  individuals  within  his  organization  on  the 
same  program  was  provided  to  him.  The  estimates  made  by 
other  organizations  relative  to  the  same  program  were  not 
routed  back  to  him.  A few  estimators  in  the  5?Cs  took  it 
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upon  themselves  to  attempt  tracking  their  own  records,  but 
this  was  the  exception  rather  than  the  rule.  In  these  few 
cases,  it  was  individual  initiative  and  not  official 
organization  policy  to  do  so.  Further,  the  estimator  was 
not  rewarded  by  the  organization  for  this  extra  effort 
(^7i  50:  52).  An  estimator  in  one  organization  reported 
that  his  organization  did  attempt  to  keep  a track  record  of 
estimates.  However,  he  indicated  that  the  record  was  kept 
by  program  and  did  not  identify  the  individual  making  the 
estimate.  He  further  stated  that  the  information  kept  was 
unofficial . 

6.  All  cost  estimators  generally  indicated  that  it 
was  highly  important  to  generate  an  accurate  estimate. 
However,  there  was  virtually  no  consensus  from  organization 
to  organization  (or  from  estimator  to  estimator  for  that 
matter)  on  how  to  define  accuracy.  The  response  to  the 
question  "What  do  you  mean  by  accurate?"  ranged  frcm  "Plus 
or  minus  25/^  of  final  total  cost."  to  "It  varies  from  phase 
to  phase."  to  "It's  a relative  thing."  to  "How  close  the 
estimate  you  make  comes  to  the  final  cost  for  the  system." 
Many  interviewees  could  not/would  not  attempt  to  provide  a 
definition  of  the  term  accurate--they  responded  instead  with, 
"I  don't  know."  'When  the  measurement  of  accuracy  was 
pursued,  the  question  of  whether  or  not  it  would  be  most 
appropriate  to  use  actual  dollar  differences  between  the 
estimated  and  actual  costs  or  to  use  percentage  fig'ures  left 
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some  estimators  perplexed  and  uncertain.  Many  would  prefer 
to  provide  a cost  range  rather  than  a point  cost  estimate. 


Almost  without  exception  though,  each  estimator  indicated 
that  the  estimates  generated  at  his  level  of  the  estimating 
process  provided  the  most  accurate  estimates  (32;  37;  38;  39; 

43:  44;  46;  47:  49;  5O;  51;  52).  Without  a standardized 

definition  of  what  "accuracy"  means;  it  is  difficult  at  best  ji 

t 

to  determine  if  any  of  the  estimates  reflect  actual  program  jl 

I 

costs,  let  alone  which  estimate  is  most  accurate.  'i 

One  reason  frequently  given  by  many  of  the  | 

estimators  for  track  records  not  being  maintained  by  the 

organization  are  the  continual  changes  that  occur  in  each  1 

program.  This  reason  was  also  put  forth  as  the  cause  for  ,j 

not  knowing  how  accurate  the  individuals  cost  estimates  i] 

ri 

are.  The  estimators  perceived  this  to  be  a major  hindrance  ( 

to  providing  accurate  estimates  for  the  program.  Again,  ;| 

I' 

only  a few  estimators  took  the  time  to  go  back  to  their  i| 

original  estimates  and  update  them  to  reflect  the  new  ] 

information  resulting  from  program  changes.  i 

The  authors  recognize  that  changes  in  program  |j 

i; 

requirements  and  weapon  system  designs  do  occur,  and  that 

i| 

these  changes  do  represent  a major  problem  to  the  indivicual  !] 

estimator.  This  is  not,  however  a valid  reason  for  the  j 

organization  not  attempting  to  keep  individual  track  records, 
or  not  maintaining  a data  base  which  reflects  the  changes 
in  past  estimates  resulting  from  these  program  changes. 
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Nor  should  these  program  changes  confuse  the  issue  of 


accuracy.  There  are,  after  al_ , many  more  stable  aspects 
of  a program  than  there  are  changes  for  the  individual 
estimator.  It  may  well  be  that  the  estimator  should  not 
be  expected  to  be  100!^  accurate,  particularly  when  program 
changes  effect  the  item.s  estimated.  This  does  not,  however, 
reduce  the  value  of  an  accepted  measure  of  accuracy  and  a 
form,al  system  of  feedback  to  provide  accuracy  data.  Only 
by  comparing  the  accuracy  of  one  set  of  estimates  to 
another  can  any  real  information  be  generated  on  what 
estimating  techniques  or  models  provide  the  most  useful 
estimates  for  specific  types  of  weapon  systems  in  various 
weapon  system  program  phases. 

Summary . Within  the  AF  cost  esti.mating  process,  cost 
estimates  are  developed  in  an  atmosphere  of  optimism.  The 
cost  estimator  is  provided  organizational  motivation  to 
develop  an  estimate  which  can  be  used  to  advocate  the  new 
weapon  system  to  the  DOE  and  Congress.  The  individual 
estimator  is  hampered  in  his  efforts  to  develop  estimates 
by  the  lack  of  an  adao^uate,  standardized  data  base;  lack  of 
feedback  on  the  accuracy  of  his  estimates!  and  little  or 
no  feedback  on  how  changes  have  effected  prior  estimates. 
Further,  while  the  esti.mators  recognised  the  importance  of 
"accurate"  estim.ates,  the  authors  could  find  no  concensus  as 
to  what  "accuracy"  means. 
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CHAPTER  ’/ 


CONCLUSIONS  AND  REC0r»LV:ENDATI0N3 


S ummarv 

The  purpose  of  this  thesis  was  to  investigate  the 
cost  estimation  process  within  the  Air  Force.  The  study 
identified  the  organizations/agencies  which  make  cost 
estimates  and  the  points  in  time  when  they  do  so.  The  amount 
of  data  available  for  making  cost  estimates  and  the  techniques 
employed  to  generate  the  estimates  were  also  identified.  The 
information  collected  in  this  investigation  can  be 
valuable  in  understanding  the  cost  esti.mating  process 
v/ithin  the  AF.  It  will  also  provide  some  insight  into  the 
quality  and  the  limitations  inherent  in  the  Air  Force's 
system  for  generating  cost  estimates  on  new  major  weapon 
system  acquisitions. 

The  thesis  was  organized  into  a two  phased  effort: 

1.  to  build  a model  of  the  cost  estimating  process 
for  new  major  AF  weapon  systems  from  available  literature 
sources,  and 

2.  to  demonstrate  whether  the  model  could  be 
validated  as  realistically  reflecting  the  real  world 
estimating  environment  within  the  AF  weapon  system  acquisition 
process.  If  not,  to  develop  a model  reflecting  the  actual 

AF  estimating  environment. 
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An  extensive  literature  review  was  conducted  to 
build  the  original  model.  Four  phases  of  the  weapon  system 
acquisition  process  (the  conceptual,  validation,  full  scale 
development,  and  production  phases)  were  identified  as 
periods  during  which  Air  Force  and  Department  of 
Defense  organizations/agencies  make  cost  estimates. 
Therefore,  the  weapon  system  aca^uisition  cycle  was  used  as 
the  basic  framework  upon  which  the  models  v/ere  built. 
Approximaxely  SO  individuals  involved  in  cost  estimating, 
from  the  SFO  level  through  the  DCD  level,  cooperated  in 
extensive  interviews  during  the  validation  effort. 

Conclusions 

A model  of  the  Air  Force  cost  estimating  process 

for  new  weapon  system  acquisitions  was  constructed  from  the 

literature  available.  The  model  is  presented  in  figure  3 

(page  So).  This  model  depicts  what  the  concerned  and 

industrious  reader  can  interrupt  from  the  available 

literature  relative  to  the  Air  Force's  cost  estimating 

process  for  major  weapon  system  acquisitions.  The  criteria 

established  for  this  research  effort,  required  that  the  real 

world  estimating  environment  match  the  literature  model 

lOO/o  of  the  time  for:  (l)  when  an  organization/agency  wculd 

make  an  estimate;  and  (2)  what  techniques  the  organization/ 

agency  actually  used.  The  literature-based  model  did  not 

realistically  reflect  the  real  v/orld  estimating  environment 

when  tested  against  this  criteria.  The  cost  estimating 
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techniques  used  by  the  5?0  and  ICA  estimators  in  the 
validation  and  full  scale  development  phases  of  xhe  weapon 
system  acquisition  process  did  not  support  the  model.  The 
real  world  estimating  environment,  based  on  acxual  field 


interviews,  is  depicted  in  figure  4 (page  83).  It  is 


concluded,  therefore  that  the  available  literature  does  not 


accurately  reflect  the  "real  world"  of  cost  estimation  for 
j major  AF  weapon  systems  acquisitions. 

It  was  found  that  the  AF/DOD  does  not  have  an 
: adequate  definition  of  an  "accurate"  estimate.  Further, 

' the  Air  Force  does  not  even  recognize  a standardized  method 

of  measuring  accuracy.  This  deficiency  has  led  to  confusion 
among  the  individual  cost  estimators. 

The  feedback  system  for  estimating  accuracy  v/ithin 
the  AF  is  totally  inadequate  for  the  individual  estimator's 
needs.  Specifically,  no  structured  feedback  system  exists. 
The  limited  amount  of  feedback  available  is  not  standardized 


or  tailored  to  the  individual  estimator.  If  the  estimator 


is  to  improve  his  estimating  ability,  he  must  be  provided 
consistent,  standardized  reports.  Preferably  this  will 
occur  on  an  individualized  basis  and  will  indicate  how  well 


he  has  done  in  his  previous  estimating  efforts. 

The  authors  concluded  that,  in  the  majority  of 
cases,  the  organizations/agencies  involved  in  cost 
estimating  for  new  major  AF  wea.pon  systems  act  in  a program 
advocacy  role.  It  appears  the  ICA  program  as  currently 
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irnplementecl  does  not  produce  a truely  independent  cost 
estimate  as  originally  intended.  AFSC,  which  conducts  the 
majority  of  the  ICAs , uses  the  Directorate  of  the  Comptroller 
of  the  division  (ASD,  SSD,  or  5AI»ISC ) to  v/hich  the  program 
is  assigned  to  conduct  its  ICA.  Thus,  the  same  individual 
(division  commander)  is  basically  responsible  for  producing 
both  the  ICA  and  the  S?0  estimates.  The  authors  perceive  it 
as  unlikely  that  under  these  circumstances,  widely  differing 
estimates  could  possibly  be  generated  and  forwarded  for 
comparison. 

The  majority  of  cost  estimates  made  within  the  AF 
cost  estimating  process  are  optimistic,  and  are  apparently 
based  on  an  optimal  contractor  and  Air  Force  level  of 
performance  in  the  development,  production  and  operation  of 
the  new  weapon  system.  These  estimates  ostensibly  result 
from  the  program  advocacy  roles  assumed  by  the  majority  of 
AF/DCD  persorucel  involved  in  weapon  system  acquisitions. 

It  is  apparent  that  over  timie,  such  optimistic  cost 
estimates  can  cause  the  Congressional  and  Executive  branches 
of  the  government  to  question  the  credibility  of  the  Air 
Forces  estimating  system. 

A final  conclusion  reached  was  that  the  estimators 
do  not  agree  among  themselves  on  the  terminology  to  use  in 
describ ing'^naming  the  available  estimating  techniq.  es,  what 
they  involve,  and  into  which  category  a particular  technique 
should  be  placed.  In  fact,  agreem.ent  could  not  be  fo'und  as 
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to  how  many  categories  of  techniques  there  really  are. 

In  summary,  the  material  gathered  in  this  study  leads 
xhe  authors  to  believe  that  AF  organicaxions  from  che  5?C 
level  up  to  and  including  the  DCD  do  not  create  the  conditions 
necessary  to  motivate  estimators  to  develop  estim.ates  which 
truly  reflect  final  total  system  costs  for  new  major  weapon 
systems.  It  should  be  emphasised  that  the  authors  are 
speaking  of  new  weapon  systems  where  little  "hard-core" 
production  data  is  available.  The  individual  estimator  is 
without  an  adequate  data  base  from  which  to  better  the 
accuracy  of  his  estimates:  he  is  not  provided  feedback  on 

the  accuracy  of  his  estimates,  and  how  changes  which  have 
occurred  effect  prior  program  estimates;  he  is  provided  no 
feedback  on  hew  his  individual  estimates,  which  are  part  of 
the  total  estimate  forwarded  to  higher  level  agencies 
compare  with  previous  program  estimaxes--his  own,  these  made 
by  others  within  his  organization,  or  xhose  made  by 
estimators  from  outside  his  organization;  and  he  is  not 
provided  with  data  on  what  the  program  final  costs  were  cr 
how  his  estimate  compared  with  them  (normally  by  the  time  the 
final  costs  for  a program  are  available,  the  individual 
estimator  is  no  longer  associated  with  the  program) . The 
individual  estimator  is  internally  motivated  to  provide  an 
optimistic  estimate  which  can  be  used  to  advocate  the  weapon 
system  to  Congress. 

Finally,  while  the  estimators  indicated  they  felt  it 
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was  critical  that  the  estimate  be  "accurate”,  there  is  no 
consensus  as  to  what  "accuracy"  means.  A rehetorical  question 
might  be  appropriate  at  this  point.  "What  motivates  the 
individual  estimator  to  generate  an  accurate  estimate  if  he 
doesn't  know  what  an  accurate  estimate  is?"  Further,  "Hov/ 
can  an  estimator  be  expected  to  improve  his  estimates  over 
time  when  he  receives  no  feedback  to  indicate  the  type  or 
amount  of  error  which  exists  in  his  earlier  estimates?" 

The  results  of  this  analysis  thus  led  an  unbiased  observer 
to  question,  "What  should  be  the  AF's  standardised  and 
measurable  definition  of  accuracy?" 

Recommendations 

A publication  that  shows  the  actual  relationships 
betv/een  the  organizations/agencies  involved  in  the  cost 
estimating  process  is  required.  This  publication  could 
be  used  to  indoctrinate  new  individuals  entering  the  cost 
estimating  field.  It  could  also  be  used  to  orient 
individuals  currently  functioning  in  the  cost  estimating 
field  on  how  all  facets  of  the  system  im.part  the  final 
estim.ate  generated  on  a weapon  system.  Such  a publication 
would  depict  where  the  specific  organization/agency  fits  into 
the  overall  AF  cost  estimating  process.  This  thesis  could 
be  used  as  a basis  for  the  publication.  If  the  14 
organizations/agencies  interviewed  in  this  study,  which  are 
involved  in  making  cost  estimates  for  new  weapon  acquisitions, 

12  requested  copies  of  the  final  thesis.  They  expressed  a 
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hope  that  the  thesis  would  provide  an  insight  into  the  cost 
estimating  process  which  could  benefit  their  estimators. 
Unfortunately,  this  thesis  will  not  carry  the  weight  of  an 
official  AF  document.  Further,  the  organization  of  a thesis 
is  not  appropriate  for  providing  the  information  in  an 
efficient  package  for  use  by  these  organizations.  A logical 
extension  of  this  thesis  would  result  in  an  approved 
compendium  of  cost  estimating  techniques  which  could 
establish  a standardized  AF  terminology.  This  compendium, 
when  developed,  should  be  included  in  any  publication 
developed  to  depict  the  AF  cost  estimating  process. 

The  AF  needs  to  establish  a feedback  system  which 
identifies  the  individual  generating  the  estimate,  the  phase 
of  the  weapon  system  acquisition  cycle  during  which  the 
estimate  was  generated,  and  the  techniques/methodology  used 
in  generating  the  estimate,  all  in  a standardized  and 
comparable  format  across  weapon  systems.  Developing  and 
implementing  this  feedback  system  would  facilitate  the 
development  of  a standardized  data  base  for  use  by  all 
estimators  when  working  on  the  cost  estimates  of  future 
weapon  system  acquisitions. 

The  AF/DOD  should  begin  to  manage  the  cost  estimating 
process  as  an  entity  in  itself  with  the  objective  of 
improving  cost  estimating  accuracy.  To  do  this,  four  actions 
as  a minimum  are  required: 

1.  Establish  a consistent  measure  of  accuracy  based 
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upon  a standardized  definition  of  the  term. 

2.  Develop  and  implement  a feedback  system  based  on 
the  individual  estimator  and  organizational  program 
estimates . 

3.  Free  the  estimator  from  his  percep-::ion  of  a need 
to  produce  optimistic  estimates  based  upon  an  optimal 
performance  in  all  facets  of  the  program. 

4.  Develop  and  implement  a positive/negative  reward 
system  based  upon  obtaining  accurate  estimates. 

Recommendations  for 
Future  Studies 

Based  upon  the  absence  of  any  concensus  about  the 
terminology  of  cost  estimating  techniques  in  the  literature 
reviewed,  and  backed  by  the  same  finding  among  the  individual 
estimators  interviewed,  the  authors  strongly  recommend  that 
future  research  be  conducted  to  develop  a compendium  cf  the 
techniques  used  in  cost  estimating.  The  study  should 
identify  the  techniques  , explain  hov/  the  techniques  are 
used,  and  standardize  the  terminolog;^'  relevant  to  each 
particular  technique.  Hopefully,  the  rationale  for  a 
categorization  of  the  techniques  could  be  developed  with 
assistance  from  all  levels  within  the  AF  cost  estimating 
process  for  inclusion  in  the  compendium.  This  compendium, 
when  completed,  should  be  included  in  the  AF  document 
recommended  earlier. 

Based  upon  the  conclusion  that  the  ICA  program  may 
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not  be  functioning  as  originally  intended,  further  research 
into  the  independence  and  use  of  the  ICA  estimate  appears 
warranted.  One  of  the  possibilities  to  be  investigated 
would  be  whether  the  ICA  should  be  conducted  by  an  agency 
completely  divorced  from  the  MAJCOM  which  produces  the  3FC 
estimate  or  perhaps  by  an  organisation  totally  independent 
from  DOD! 


Further  research  should  be  conducted  to  determine  the 
design  and  content  requirements  for  a standardized  data 

base.  The  currect  data  bafSis  shcM||b|^^e  analysed  to  determine 

# \ 

their  shortcomings  as  well  as  their  beneficfl^  features. 


yolve  fhe  individual  estimators  to 
ts/desires  relative  to  the 


The  researi 
ascertain ' 
information  needed. 

A final  area  which  requires  further  research  is  the 
motivation  of  the  individual  estimator.  Extensive  research 
should  be  accomplished  to  identify  the  factors  which  motivate 
the  estimator  to  generate  good  estimates.  The  pros  and 
cons  of  a reward  system  based  upon  the  measured  accuracy  of 
the  individual  estimator  and/or  his  organization  should  be 
included . 


The  cost  estimating  process  within  the  AF  is 
typified  by  confusion  and  uncertainty  in  what  is  involved 
and  how  it  works.  This  confusion  and  uncertainty  is  found 
among  the  estimators,  those  who  act  upon  the  estimate  made, 
and  at  the  Air  Staff/DCD  level.  Additional  research  is 
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necessary  to  further  explain  the  process  and  thereby  reduce 
the  confusion  and  uncertainty.  Air  Force  publications  are 
needed  to  orient  both  new  and  experienced  estimators  to  the 
realities  of  the  Air  Force  estimating  process.  A better 
understanding  of  the  cosc  estimating  process  is  necessary  i: 
the  AF/DCD  is  to  reduce  "apparent"  cost  growth  and  regain 
public  and  Congressional  confidence  in  their  ability  to 
estimate  accurately  the  costs  of  new  weapon  systems. 
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APPENDIX  A 


JlANAGERIAL  INTERVI3V  SCHEDULE 
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:4ANAGERIAL  INTERVIEJ\f  SCHEDULE 


Type  Interview, 

Tape  Number 

Date 


I.  General: 

A.  Personal  and  Organizational  Data: 

1 . Name? 

2.  Crganization/Division? 

3.  Job  Title? 

L.  Length  of  time  in  current  position? 

5.  Length  of  time  as  a cost  estimator? 

B.  Purpose  of  Study:  To  clarify  and  document  the 

overall  cost  estimating  process  used  by  the  Air 
Force  in  the  acquisition  of  major  new  weapon  systems. 

C.  Purpose  of  Interview:  To  obtain  your  help  in  under- 

standing what  organizations/agencies  provide  cost 
estimates,  the  purpose  for  which  the  cost  estimates 
are  made,  when  in  the  acquisition  process  these  cost 
estimates  are  generated,  and  what  techniques  are 
appropriate  to  each  organization/agency  in  producing 
a cost  estimate. 

II.  Specifics: 

A.  When  are  cost  estimates  generated? 
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1.  During  which  phase  of  the  weapon  system 


acquisition  process  does  your  organization  make 
cost  estimates? 

a.  Conceptual?  

h.  Validation?  

c.  Full-Scale  Development?  

d.  Production?  

2.  Can  you  differentiate  between  the  phases  when 
you  make  a cost  estimate? 

3.  How  do  you  differentiate? 

4.  Do  you  make  more  than  one  cost  estimate  during 
any  one  phase? 

a.  Which  phase? 

b.  How  many  estimates  are  made? 

c.  Why  do  you  make  more  than  one  estimate? 

d.  How  are  they  different  one  from  another? 

3.  Amount  of  Data  Available. 

1.  How  much  data  is  available  for  making  cost 
estimates  in  each  of  the  following  phases? 

a.  Conceptual? 

b.  Validation? 

c.  Full-Scale  Development? 

d.  Production? 

2.  What  type  data  do  you  use  to  make  cost  estimates 
in  each  of  the  following  phases? 

a.  Conceptual? 
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b.  Validation? 

c.  Full-Scale  Development? 

d.  Production? 

3.  What  data  would  be  required  to  make  cost 
estimating  easier? 

a.  Why  would  this  data  be  reo.uired? 

b.  How  would  this  additional  data  help? 
'Techniques  used  to  make  cost  estimates. 

1.  What  techniques  are  available  to  you  to  make 
cost  estimates? 

2.  What  techniques  do  you  use  to  make  cost 
estimates  in  each  of  the  following  phases? 

a.  Conceptual? 

b.  Validation? 

0.  Full-Scale  Development? 
d.  Production? 

3.  Do  you  feel  that  the  technique (s)  currently  used 
in  each  phase  is  the  most  useful  one  available? 

a.  Why/why  not? 

b.  What  techjoiques  would  be  mors  useful? 

c.  Why  do  you  feel  the  techniques  you 
suggested  would  be  more  useful? 

4.  Do  you  feel  that  the  techniques  currently  used 
in  each  phase  provides  the  besc  estim.ate 
possible? 

a.  Why/why  not? 
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b . What  techniques  would  provide  a more 
accurate  estimate? 

c.  Why  do  you  feel  that  the  technique  suggested 
would  be  betcer  than  the  one  currently  used? 

d.  How  accurate  are  your  estimates? 

e.  Can  you  tell  how  accurate  your  estimates 
are? 

f.  How  can  you  tell? 

5.  Do  outside  organicaoions/agencies  attempt  to 

influence  you  in  the  choice  of  the  techniq,ue  to 
use? 

a.  Who? 

b.  When? 

c . Why? 

D.  Why  are  cost  estimates  made? 

1.  Why  do  you  make  cost  estimates  in  each  of  the 
four  phases? 

a.  Conceptual? 

b.  Validation? 

c.  Full-Scale  Development? 

d.  Production? 

2.  Who  uses  the  cost  estimate  made  by  you  in  each 
of  the  four  phases? 

a.  Conceptual? 

b.  Validation? 


c.  Full-Scale  Development? 


Production? 
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3.  What  are  the  cost  estimates  used  for? 


a.  Conceptual: 

( 1 )  . Planning?  

(2) .  Controlling?  

(3) *  Other?  

b.  Validation: 

( 1 )  . Planning?  

(2) .  Controlling?  

(3) .  Other?  

c.  Full-Scale  Development: 

(1 )  . Planning?  

(2) .  Controlling?  

(3) .  Other? 

d.  Production: 

(1) .  Planning?  

(2) .  Controlling?  

(3) *  Other?  

Importance . 

1.  How  important  do  you  feel  it  is  that  the  cost 
estimate  you  generate  be  accurate? 

2.  What  do  you  mean  by  accurate? 

3.  How  do  your  estimates  compare  with  those  made 
in  other  organizations? 

U,  Do  outside  organizations/agencies  attempt  to 
influence  the  outcome  of  your  estimate? 
a.  Who? 
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ORGANIZATIONAL  LISTING  OF  INTERVIEW2ES 


The  following  individuals,  from  the  organizations 
listed,  were  interviewed  in  the  process  of  validating  the 
model  developed  in  Chapter  II  and  updated  in  Chapter  IV. 


AF  CAIG  (HQ  AF/ACMC) 

1.  Fitzgerald,  Nia^or  Dan  (28  & 29  March  77) 

2.  Krushinski,  Joseph  (28  & 29  March  77) 

3.  Puryear,  Captain  Franklin  (28  March  d 19  April 
77) 


AFLC  Comotroller  (HQ  AFLC/ACRC) 


1.  Jones,  Captain  Charlie  E.  (9  February  77) 

2.  Waker,  Michael  ?.  (3  February  77) 

3.  Wallace,  John  M.  (4  February  77) 


AFLC  Flans  (HQ  AFLC) 


1.  Telford,  Lt  Col  'Jilliam  D.  (XRXXS)  (23  March  & 
28  April  77) 

2.  Dellinger,  Edward  K.  (MMA)  (28  April  77) 


AFLC  ALC 

1.  Williams,  William  D.  (WR-ALC/M»iH( 2 ) ) (29  April  77) 

2.  Davis,  Larry  (CC-ALC/MMHA)  (6  Sc  10  May  77) 

3.  Poe,  Lt  Col  Joseph  (CC-ALC/M.ZHA)  (6  May  T"’) 


AFSC  Comrtroller  (HQ  AFSC/ACCE) 

1.  Scarlett,  Lt  Col  Bobby  R.  (28  & 29  March  77) 

2.  Sims,  Captain  Sherry  (28  & 29  March  77) 

3.  Rasberry,  Dayle  H.  (29  March  & 10  May  77) 

AFSC  Plans  (HQ  AFSC/XRX) 

1.  Blake,  Lt  Col  Thomas  (4  February  3c  29  March  77) 

2.  Bando,  Edward  (I9  May  77) 

3.  Sink,  Captain  John  (I9  May  77) 


AFSC  ASP  Comptroller  (ASD/ACC) 

1.  Adams,  Charles  W.  (ACC)  (10  January  77) 

2.  Richey,  Lawrence  S.  (ACCC)  (2^  January  77) 

3.  Sampson,  Charles  E.  (ACCC)  (8  February  77) 

E.  Thorpe,  Thomas  E.  (ACCC)  (24  January  77) 

AFSC  ASP  System  Program  Offices 

ACCX-Advanced  Systems  Cost  Division 

1.  Burnett,  Robert  V.  (31  January  77) 

2.  Fetter,  Donn  P.  (11  February  77) 

3.  Schloaser,  Kelsey  P.  (11  February) 

4.  Weiler,  Henry  F.  (2  February  77) 

SD-28P-Airlif t Systems 

1.  Ambrose,  John  R.  (IS  January  77) 

2.  C shorn,  Donald  A.  (3  February  77) 

3.  Wethington,  Clyde  M.  (27  January  77) 

5D-65-Maveric  Program 

1.  Schwenke,  Captain  Robert  B.  (31  January  77) 

YFPR-F15 

1.  Conley,  Robert  (8  March  77) 

2.  Kouri,  Donna  (19  Miarch  '^'^) 

YHPF-Bl 


1.  Clapper,  Capxain  '.villiam  (10  February  77) 

2.  Townsley,  ustin  L.  (16  February  77) 

3.  Frye,  Thomas  L.  (16  February  ■’7^ 

YMPM-Air  Launched  Cruise  Missile  Program 

1.  Allen,  John  A.  (3I  January  ^7) 

2.  Mialhotra,  Pran  L.  (21  January  77) 

3.  Wagner,  Joseph  T.  (21  January  77) 

YPKF-FI6 

1.  Jack,  Ivlajor  James  (3  February  77) 

2.  Thompson,  Rick  (24  March  77) 

3.  Ambrose,  John  J.  (24  l’<arch  77) 

4.  Thorp,  Thomas  E.  (24  .‘•a.rch  77) 
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YXPC-AIO 


1.  Murchand,  Richard  C.  (2  February  77) 

2.  Bigood,  Leroy  (2  February) 


AFSC  SSD  Comptroller  (ESD/ACCE) 

1.  Coakly,  Ellen  M.  (22  February  77) 

2.  McNeil,  Hugh  W.  (24  February  77) 

3.  Wisialko,  Albert  J.  (23  February  77) 

AFSC  ESP  System  Program  Offices 
YSXF- Airborne  Command  Post 

1.  Seigel,  Robert  (25  March  77) 

2.  Melanson,  Robert  (25  ferch  77) 

YWXF-Airborne  Warning  and  Control  System 

1.  Patrick,  Major  James  (18  March  77) 

2.  C-rones , Captain  Royce  (11  March  77) 

3.  Roberts,  Captain  Kenneth  M.  (11  March  7?) 

AFSC  SAJ^ISC  Comptroller  (ACCE) 

1.  Cook,  George  M.  (18  February  77) 

2.  Green,  J.  Richard  (19  January  77) 

3.  Tomlinson,  Roberx  L.  (18  February  77) 

4.  To’/msend,  Lt  Richard  L.  (17  February  77  ^ 
March  77) 

AFSC  SAIvSC  System  Program  Offices 

ACCE-Comptroller  (in  support  of  SPCs) 

1.  Fong,  Franklin  K.  (24  March  77) 

2.  Martin,  Martha  A.  (17  February  77) 

3.  Wilcox,  Leonard  (24  February  77) 

LV-Launch  Vehicles 

1.  Kelley,  Lt  Col  Gilbert  F.  (3  March  7?) 

2.  Kisko,  Captain  William  A.  (7  March  77) 

3.  Pearse,  Major  James  P,  (10  February  7^) 

MNPC-ICEM  Program  Office 

1.  Wall,  William  E.  (4  March  77) 
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SKP-Space  Communications 


1.  McCoy,  Lt  Col  Donald  G.  (18  February  7?) 
YDC-Defense  Meteorological  Support  Program 

1.  Clovicko,  Lt  Cary  (11  February  77) 


YRP-Satellite  Data  Systems 


1.  Figueroa,  Captain  Francisco  A,  (10  February  77) 
HQ  Aerosnace  Defense  Command  (ADCOM) 

1.  Behrends,  James  L.  (ADCOM/ACBA)  (1  February  77) 

2.  Smith,  Major  Bruce  E.  (ADCCM/XPX)  (l  February  77) 

3.  Whitaker,  Captain  Richard  B.  (ADCOM/XPDS)  (8 
February  77) 

HQ  Military  Airlift  Command  ( MAC ) 


1.  Nelson,  Captain  Eric  E.  (MC/LGXP)  (3  January  77) 

2.  Stauffer,  Captain  Christian  VJ . (ivlAC/ACMC)  (10 
February  77) 

HQ  Strategic  Air  Command  (SAC) 

1.  Ellinger,  Captain  Robert  W.  (3AC/XPQR)  (L 
February  77) 

2.  Wade,  Mia j or  Tommy  H.  SAC/ACMC)  (U  February  77) 

HQ  Tactical  Air  Command  (TAC) 

1.  Buttross , Captain  David  A.  (TAC/ACMC)  (14 
February  77) 

2.  Whelan,  Major  Charles  (TAC/DRDR)  (9  February  77) 
CSD  CAIG  (CSD  (P&E)) 

1.  Pilling,  Lt  Cmdr  Donald  (28  & 29  J'iarch  77) 

2.  Manetti,  Howard  J.  (29  March  77) 
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APPENDIX  C 
GLOSSARY  OF  TERMS 


Advocacy  Package;  The  documents  necessary  to  present  the 


conclusion  of  the  Conceptual  Phase  study  and  the  various 
conceptual  alternatives,  along  with  the  studies  and  tests  of 
these  alternatives.  The  package  recommends  one  particular 
alternative  concept.  The  advocacy  package  is  used  to  "sell" 
the  program  to  CSD. 

Budget  Cost  Information;  Information  on  proposed  alternative 
solutions,  costs,  and  schedules  for  satisfying  a RCC. 

Normally  prepared  by  AFSC  and/or  AFLC  in  response  to  a 
published  ROC. 

Budget  Costs;  Costing  used  in  budget  submissions  as 
distinguished  from  costing  used  in  program  documents. 

Budget  costs  represent  the  specified  total  obligation 
authority  requirements  for  funds  in  a particular  fiscal 
period  and  generally  represent  a refinement  of  program  costs. 
Configuration  Item;  An  aggregation  of  hardware/software,  or 
any  of  its  discrete  portions,  which  satisfies  an  end  use 
f’unction  and  is  designated  by  the  Goverrjnent  for  conf ig'uration 
management.  During  development  and  initial  production,  Cl's 
are  only  those  specification  items  that  are  referenced 
directly  in  a contract  (or  an  equivolent  in-house  agreement). 
Cost  Analysis  Im-provement  Group  (CAIG) ; An  CSD  (AF)  advisory 
body,  responsive  to  the  DSARC  (AFSARC)  on  matters  relating  t 


0 


cost.  The  CAIG  provides  the  DSARC  (AFSARC)  with  a review 
and  evaluation  of  independent  (IGA)  and  program  (G?C)  cost 


estimates  prepared  hy  the  I*Iilitary  Services.  Established 
to  develop  uniform  criteria  to  be  used  by  all  DCD  units 
making  cost  estimates. 

Cost  Category;  One  of  three  types  of  costs  into  which  the 
total  cost  of  a program  element  is  divided  (1)  research 
and  development,  (2)  investment,  and  (3)  operations. 

Cost  Estimate:  The  product  of  an  estimating  procedure 

which  specifies  the  espected  dollar  cost  to  perform  a 
stipulated  task  or  to  acquire  an  item.  It  may  be  stated 
as  a single  value  or  range  of  values. 

Cost  Est imaging  Relationship  (CER);  An  analytical  expression 
which  describes,  for  predictive  purposes,  the  quanity  or 
cost  of  an  item  or  activity  (either  in  physical  units  or 
dollars)  as  a function  of  one  or  more  explanatory  variables. 
Defense  System  Acquisition  Review  Council  (DSARC);  An 
advisory  body,  within  the  Cffice,  Secretary  of  Defense, 
which  is  chartered  to  review  and  evaluate  the  status  of 
each  appropriate  system  acquisition  program  at  three  basic 
milestone  points.  Membership  is  composed  of  The  Director, 
Defense  Research  & Engineering  (DDR&E) , the  Assistant 
Secretary  of  Defense  (Comptroller  (ASD  Comptroller)  ),  the 
Assistant  Secretary  of  Defense  (Installations  and  Logistics 
(ASD  (I3cL))  and  the  Assistant  Secretary  of  Defense  (System.s 
Analysis)  (ASD  (3A)).  The  three  milestone  points  in  the 
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weapon  system  acquisition  process  which  require  DSARC  review 
and  evaluation  are: 

1.  When  initiation  of  Contract  Definition  (or 
equivalent  effort)  is  proposed; 

2.  When  transition  from  the  Contract  Definition  Phase 
to  full  scale  development:  and 

3.  When  transition  from  the  development  phase  into 
production  for  Service  deployment  is  proposed. 

Development  Concept  Parer  (DC?):  Kemoranda  from  the 

Secretary  of  Defense  expressing  his  decisions  on  the 
initiation  of,  or  changes  to,  major  H<iD  programs. 

Five  Year  Defense  Plan  (FYDP):  The  official  program  which 

summarizes  the  Secretary  of  Defense  approved  plans  and 
programs  for  the  DOD. 

Independent  Cost  Estimate  (ICE);  An  estimate  of  program 
cost  developed  outside  normal  advocacy  channels  hy  a 
team  which  generally  includes  representation  from  Cost 
Analysis,  Procurement,  Production  Management,  Engineering 
and  Program  Management.  Synonomous  with  Independent  Cost 
Analysis  (ICA)  in  this  paper.  An  ICA  generally  involves 
maximum  use  of  parametric  techniques  but  may  use  other 
cost  estimating  techniques. 

Industrial  (Detailed)  Engineering  Estim.ate;  A ccst  estim.ate 
based  on  manufacturing,  assembly,  and  test  costs  generated 
from  system  description  operations,  or  standards  designed 
from,  time  and  motion  studies  or  vender  quotes.  Also  1-cnown 
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as  industrial  buildup  or  grass  roots  estimates. 

Official  Program  Estimate;  The  estim.ate  prepared  by  a 
System  Program  Office  of  Aeronautical  Systems  Division, 

Electronics  System  Division  or  Space  and  Missile  System 
Division, 

Parametric  Post  Estimate;  A cost  estimate  which  is  based  cn 

the  development  and  utilization  of  estimating  relationships  i 

1 

between  historical  costs  and  other  progra.m  variables  such 
as  system  physical/performance  characteristics,  contractor 
output  measures,  manpower  loading,  and  facility  floor  space. 
Plarning/Programming.^udgeting  System  (PPBS);  An  integrated 
system  for  the  establishment,  maintenance,  and  revision 
of  the  FYDP  and  the  DOD  budget. 

Program  I<Ianagemer.t  Directive  (PMD);  A USAF  document  which  i 

transmits  to  AFSC  the  "go-ahead"  and  guidance  for  the  new 
weapon  system  following  the  DSARC  I decision. 

Recuired  Orerational  Oanabilitv  (ROC);  A formal,  numbered 
document,  used  to  identify  an  operational  need  and  to 
req_uest  a new  or  im.proved  capability  for  the  operating 

3 

forces.  The  capability  sought  is  described  in  terms  of 
operational  objective,  operational  environment,  support 


and  maintenance  concepts,  and  concept  of  operation. 

System  Program  Office;  An  AFSC  management  organization  set 
up  to  manage  the  acquisition  of  a new  weapon  system. 

Total  System  Cost;  Total  system  cost  encompasses  costs  for 
development,  procurement,  operation,  and  support,  and 
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where  applicable,  disposal.  Total  system  cost  is  analogous 
with  life  cycle  cost. 
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